Why does seeking to the beginning of a file to reread it see updates on Windows but not Linux?

Question

I've found an inconsistent behavior of python.

Under Windows if the file changes the program will notice. Under Linux the program will not notice.

I am using python 3.6.8 and Ubuntu 18.04.

Is this a bug or do I something wrong?

import time

if __name__ == '__main__':
    file = open('CurrentData.txt', 'r')

    while True:
        lines = file.readlines()

        print(lines)

        time.sleep(1)

        file.seek(0)

    file.close()

"Changes" how? Replacing a directory entry with a whole new file is legal on UNIX, but leaves programs that opened it earlier with a handle on the old file. — Charles Duffy, Jul 05 '19 at 14:56
@CharlesDuffy Changing by replacing the lines inside the file and save the file. — gartenkralle, Jul 05 '19 at 14:59
What do you mean, by "file changes" and "the program will notice"? How is the file changed, and what exactly is the difference you recognized? — jottbe, Jul 05 '19 at 14:59
@gartenkralle, "save the file" how? With which **specific** software? Most programs on UNIX use the create-and-rename pattern to do an atomic replacement. — Charles Duffy, Jul 05 '19 at 14:59
@CharlesDuffy Nice to know. I did with gedit and save button. :) — gartenkralle, Jul 05 '19 at 15:01
Run `stat CurrentData.txt` before and after saving in gedit, and see if the inode number changes. If it does, you know gedit saves by create-and-rename. — Charles Duffy, Jul 05 '19 at 15:02
Then you'll need to re-`open()` the file to get a handle on the new inode. — Charles Duffy, Jul 05 '19 at 15:05
@CharlesDuffy If I change the content of the file by another python program the inode number doesn't change. Isn't it? — gartenkralle, Jul 05 '19 at 15:07
Depends on how that other Python program is written, but if it's just doing `open('CurrentData.txt', 'w')`, it'll be changing the existing inode, not creating a new one. See [Atomic writing to file with Python](https://stackoverflow.com/questions/2333872/atomic-writing-to-file-with-python) for discussion on how to do it the other way. — Charles Duffy, Jul 05 '19 at 15:08
...that said, if this is actually *important* data (where you need it to be preserved through power-loss events, inconveniently-timed application failures, etc), you might want to reconsider using the in-place update approach, and go the create-and-replace route. — Charles Duffy, Jul 05 '19 at 15:10
I just tested it with `echo "something" >> testfile.txt` there it worked. This obviously doesn't change the inode. — jottbe, Jul 05 '19 at 15:10
`>>testfile.txt` is actually a special case, as is opening files with the `'a'` flag in Python; either opens with the `O_APPEND` flag, and usually (not on NFS), that's guaranteed atomic and safe (so long as each individual `write()` syscall leaves the file in a valid state when it's done). Obvs., that only works if your goal is to be adding content to the end, not changing things in-place; as soon as you do a `seek()` and `truncate()` on the write side, you've thrown away a lot of the value. — Charles Duffy, Jul 05 '19 at 15:12

Charles Duffy · Accepted Answer · 2019-07-05T15:07:10.120

The only thing that's wrong with your Python program is that it's making unfounded assumptions.

There are two different ways to change a file's contents in UNIX:

You can modify the file in-place, changing the contents of the existing inode; seek()ing back to the front and rereading will see that, so if your file were edited with this method, your existing code would work.
You can create a whole new inode, write the contents, and only after the write is successful rename() it over the old one.

That's often considered the better practice, because it means programs that were in the middle of reading your old file will retain the handle they had; they won't have surprising/inconsistent/broken behavior because the contents changed out from under them. If you do it right (which might involve fsync() calls on not just the file but also the directory it's in), a writer using this method can also ensure that in the event of a power loss, the new system will have one copy of the file or the other, but not a half-written intermediate state you can get if you truncate an existing inode and rewrite from the beginning.

If you want to handle both cases, you can't hang onto your existing handle, but should actually re-open() the file when you want to see changes.

Why does seeking to the beginning of a file to reread it see updates on Windows but not Linux?

1 Answers1