-1

For example, I have a 10GB csv file, and I want to get 1000 lines in the middle of this file, or I want to get any line of this file. Are there any open source lib that can help me ? Do I have to solve this problem base on OS?

flyingrose
  • 107
  • 2
  • 11
  • 1
    Possible duplicate of ["Large data" work flows using pandas](http://stackoverflow.com/questions/14262433/large-data-work-flows-using-pandas) – João Almeida Nov 17 '16 at 10:13

1 Answers1

1

Files are iterators, and to get items out of the middle of any iterator without storing the rest you can use islice:

from itertools import islice

with open('bigfile.txt') as bigfile:
    lines = islice(bigfile, start, end)
Alex Hall
  • 34,833
  • 5
  • 57
  • 89