Replicating "tail -f" with Python

Question

According to David Beazley's talk on generators, the following code should replicate the UNIX tail -f command:

import time
def follow(thefile):
    thefile.seek(0,2)
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

f = open('followed.txt')
lines = follow(f)

for i in lines:
    print i

If I run this in a shell, it's doing "something", and indeed it locks up the IPython notebook, but it ain't printing out the contents of followed.txt. Why so?

**Clarification**: with the code running, I open followed.txt with Vim, add a line of arbitrary text, and save it. Still nothing gets printed. — Pyderman, Jan 22 '16 at 18:37
How are you appending to `followed.txt`? The code above may not work if you are adding text to `followed.txt` using a text editor, since the text editor may not be appending to the original file -- it may be creating a new file and then renaming `followed.txt` to it... — unutbu, Jan 22 '16 at 18:41
The code you posted will work if you `f = open('followed.txt', 'a')` and call `f.write` concurrently, or (on unix) if you append using something like `echo "some text" >> followed.txt`. — unutbu, Jan 22 '16 at 18:43
I used your code in windows 7 and editting the file with notepad++ and it worked perfectly — Mr. E, Jan 22 '16 at 18:44
[How to edit a file inplace using Vim](http://stackoverflow.com/q/11043534/190597) — unutbu, Jan 22 '16 at 18:45
@unutbu Thanks for the link; indeed it had not struck me that perhaps saving in a text editor/Vim was in fact creating a new file each time. — Pyderman, Jan 22 '16 at 18:54

Haifeng Zhang · Accepted Answer · 2016-01-22T18:43:45.550

5

I tried the script, it works.

You have to make sure your input file is a growing file. If not it is hanging and expecting new growing lines.

Here's a script keep writing line with timestamp into sample.csv every 5 seconds.

import os
import time
import datetime

while True:
    os.system("echo " + "sample line with timestamp:{0}".format(datetime.datetime.now()) + " >> " + " sample.csv")
    time.sleep(5)

Use your tail -f script to read it and you will see the output.

edited Jan 22 '16 at 18:43

answered Jan 22 '16 at 18:34

Haifeng Zhang

30,077
19
81
125

See my comment above. – Pyderman Jan 22 '16 at 18:37
2

@pyderman you cannot add a line and save the file. It will create a new file with the same name – Haifeng Zhang Jan 22 '16 at 18:42
Makes sense; I wonder though how Mr E above was able to achieve it with a text editor. – Pyderman Jan 22 '16 at 18:46
@Pyderman Glad it helps. BTW, It is really good Slides, thanks for your sharing! – Haifeng Zhang Jan 22 '16 at 18:50
@haifzhan That's not always true. It depends on the editor and whether it uses copy-on-write. – Brendan Abel Jan 22 '16 at 18:51

Brendan Abel · Answer 2 · 2016-01-23T04:58:18.377

The follow() generator is only going to return lines that are written to the file after follow() was called. The seek(0,2) puts the cursor at the end of the file, and then tries to read new lines from that point on.

tail usually outputs the last 10 lines by default. If you wanted something like that

def follow(thefile):
    n_lines = 0
    # Seek to the end of the file
    thefile.seek(0,2)
    # Seek the cursor back one character at a time until you
    # reach the beginning of the file or 10 newlines are found.
    while n_lines < 10 and thefile.tell() > 0:
        # Go back one character and read it.
        thefile.seek(-1, 1)
        c = thefile.read(1)
        # Only go back 10 lines
        if c == '\n':
            n_lines += 1:
        # Reset the cursor position for the character we just read
        thefile.seek(-1, 1)

    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

Replicating "tail -f" with Python

2 Answers2