Read huge file using a 'for' loop

Question

I have a huge file, abc.txt, and wanted to read line by line, not all at once. I can do it as below:

filename="c:\abc.txt"
with open (filename, "r") as fb:
    for content in fb:
        # Do something ....

Here I am not understanding one thing. As you see, "fb" is a file pointer over here. How is the "for" loop processing the pointer directly internally even without using any readline or read function?

I just wanted to know how the for loop is working here.

Possible duplicate of [How do I read a file line-by-line into a list?](https://stackoverflow.com/questions/3277503/how-do-i-read-a-file-line-by-line-into-a-list) — DCZ, Apr 03 '18 at 10:29
1. `fb` is not a "file pointer". It is a file object. 2. File objects are generators that are built to yield a file content line by line when they are iterated over (either by a loop or with the `next` function). See https://docs.python.org/3.5/glossary.html#term-file-object and https://docs.python.org/3.5/tutorial/inputoutput.html#reading-and-writing-files — DeepSpace, Apr 03 '18 at 10:32

score 2 · Answer 1 · answered Apr 03 '18 at 10:36

There's no such thing as a 'pointer' in Python.

fb is a file object. File objects in Python are a container type and generators. They implementg the iterator protocol and when looped over (for instance using a for-loop), it produces individual lines from the file that you opened.

If you want to try and see what is happening under the hood, try this in a REPL for instance:

f = open("sometextfileyouhave.txt", "r")
file_iter = iter(f)
print(next(file_iter))
print(next(file_iter))
print(next(file_iter))
print(next(file_iter))
# ...

or even shorter:

f = open("sometextfileyouhave.txt", "r")
print(next(f))
print(next(f))
print(next(f))
# ...

Read huge file using a 'for' loop

1 Answers1