python3 file.readline EOF?

Question

I am having trouble determining when I have reached the end of a file in python with file.readline()

fi = open('myfile.txt', 'r')
line = fi.readline()
if line == EOF:  //or something similar
    dosomething()

This:

c = fp.read()
if c is None:

will not work because I will loose data on the next line, and if a line only has a carriage return I will miss an empty line.

I have looked a dozens or related posts, and they all just use the inherent loops that just break when they are done. I am not looping so this doesn't work for me.
Also I have file sizes in the GB with 100's of thousands of lines. A script could spend days processing a file. So I need to know how to tell when I am at the end of the file in Python3.
Any help is appreciated.

See this [answer](https://stackoverflow.com/a/10140333/3926995) — Chiheb Nexus, Jun 30 '17 at 21:34
@Chiheb Nexus I have already looked at that... I don't understand, because what happens if I am not at EOF and I try to read a byte, what line do I read? and am I still at the start of a line? — noone392, Jun 30 '17 at 21:36
Possible duplicate of [How to find out whether a file is at its \`eof\`?](https://stackoverflow.com/questions/10140281/how-to-find-out-whether-a-file-is-at-its-eof) — Mad Physicist, Jun 30 '17 at 21:53
@Mad Physicist thats the same answer Nexus gave and it won't work for me because then the next line doesn't contain all the data — noone392, Jun 30 '17 at 21:57
So this may be really stupid but is there anything wrong with just checking len(line) > 0 ?? — noone392, Jun 30 '17 at 21:57
An empty line will contain a `\n` as a minimum. Checking `len(line) == 0` is the same as checking `line == ''`. — Mad Physicist, Jun 30 '17 at 22:35

score 6 · Answer 1 · answered Aug 20 '17 at 19:56

I ran in to this same exact problem. My specific issue was iteration over two files, where the shorter one was only supposed to read a line on specific reads of the longer file.

As some mentioned here the natural pythonic way to iterate line by line is to, well, just iterate. My solution to stick with this 'naturalness' was to just utilize the iterator property of a file manually. Something like this:

with open('myfile') as lines:
    try:
        while True:                 #Just to fake a lot of readlines and hit the end
            current = next(lines)
    except StopIteration:
        print('EOF!')

You can of course embellish this with your own IOWrapper class, but this was enough for me. Just replace all calls to readline to calls of next, and don't forget to catch the StopIteration.

That's by far the best answer both regarding elegance and readability IMHO ! — Michael Beer, Mar 11 '19 at 16:54
This works, but exception handling is much more cumbersome than just checking whether `readline` returns an empty string. — user2357112, Dec 20 '20 at 18:57

score 4 · Answer 2 · answered Dec 18 '20 at 23:07

The simplest way to check whether you've reached EOF with fi.readline() is to check the truthiness of the return value;

line = fi.readline()
if not line:
    dosomething() # EOF reached

Reasoning

According to the official documentation

f.readline() reads a single line from the file; a newline character (\n) is left at the end of the string, and is only omitted on the last line of the file if the file doesn’t end in a newline. This makes the return value unambiguous; if f.readline() returns an empty string, the end of the file has been reached, while a blank line is represented by '\n', a string containing only a single newline.

and the only falsy string in python is the empty string ('').

It is pretty bizarre that this took 3 years to be posted. – user2357112 Dec 20 '20 at 21:42 — user2357112, Dec 20 '20 at 21:42

score 2 · Answer 3 · edited Jun 20 '20 at 09:12

You can use the output of the tell() function to determine if the last readline changed the current position of the stream.

fi = open('myfile.txt', 'r')
pos = fi.tell()

while (True):
    li = fi.readline()
    newpos = fi.tell()
    if newpos == pos:  # stream position hasn't changed -> EOF
        break
    else:
        pos = newpos

According to the Python Tutorial:

f.tell() returns an integer giving the file object’s current position in the file represented as number of bytes from the beginning of the file when in binary mode and an opaque number when in text mode.

...

In text files (those opened without a b in the mode string), only seeks relative to the beginning of the file are allowed (the exception being seeking to the very file end with seek(0, 2)) and the only valid offset values are those returned from the f.tell(), or zero.

Since the value returned from tell() can be used to seek(), they would have to be unique (even if we can't guarantee what they correspond to). Therefore, if the value of tell() before and after a readline() is unchanged, the stream position is unchanged, and the EOF has been reached (or some other I/O exception of course). Reading an empty line will read at least the newline and advance the stream position.

The documentation seems to be wrong here, as text files are opened with a 't'. For binary files (those that are opened with a 'b' in the mode string, seeks are unrestricted. For text files however, it is as stated, except for the fact that it is NOT opened with a b in the mode string — Werner Erasmus, Mar 31 '22 at 19:45

score 1 · Answer 4 · answered Jun 30 '17 at 22:07

This is a demonstrative example using f.tell() and f.read() with a chunk of data:

Assuming my input.txt file contain:

hello
hi
hoo
foo
bar

Test:

with open('input.txt', 'r') as f:
    # Read chunk of data
    chunk = 4
    while True:
        line = f.read(chunk)
        if not line:
            line = "i've read Nothing"
            print("EOF reached. What i read when i reach EOF:", line)
            break
        else:
            print('Read: {} at position: {}'.format(line.replace('\n', ''), f.tell()))

Will output:

Read: hell at position: 4
Read: ohi at position: 9
Read: hoo at position: 14
Read: foo at position: 19
Read: bar at position: 24
EOF reached. What i read when i reach EOF: i've read Nothing

Empty lines cause no problems for this code. On the other hand, it doesn't read line by line, which means it's not what the question was asking for. — user2357112, Dec 20 '20 at 19:04

score 0 · Answer 5 · answered Jun 30 '17 at 21:39

0

with open(FILE_PATH, 'r') as fi:
    for line in iter(fi.readline, ''):
        parse(line)

answered Jun 30 '17 at 21:39

Yam Mesicka

6,243
7
45
64

The questioner said they're not looping. For cases where you *are* looping, you can just do `for line in fi:`. You don't need to use `iter(fi.readline, '')`; in fact, doing that will be slower than looping normally. – user2357112 Dec 20 '20 at 18:52

python3 file.readline EOF?

5 Answers5

Reasoning

Linked