0

I'm trying to scrape data from large .txt files online using Python 3.

The data I'm after is reliably in the first 2000 characters of text, and the bandwidth/download time is making download of the entire file quite costly.

Is there any way to download only the first N characters or lines from a remote file?

Something like the following:

import urllib.request
with open(urllib.request.urlopen(myurl)) as myfile:
        head= [next(myfile) for x in range(10)]
print(head)
deseosuho
  • 958
  • 3
  • 10
  • 28
  • 5
    Please see [Only download a part of the document using python requests](http://stackoverflow.com/questions/23602412/only-download-a-part-of-the-document-using-python-requests). – alecxe Aug 04 '16 at 15:36
  • http supports range requests, but you'll have to explicitly set that up and not just accept the defaults (which is to fetch the entire url contents). – Marc B Aug 04 '16 at 15:37
  • 1
    OP never said that the answer *has* to be answered using the `requests` module. The answer on the other question assumes that the server supports byte-ranges. What if it doesn't? See http://stackoverflow.com/a/38780570/1005215 – Nehal J Wani Aug 05 '16 at 03:45

0 Answers0