Continuously read the last line of log file in python

Question

I have a file in which new lines are continuously written.

In python, I would like to continuously read the last line of this file, so that I can process the line.

I know there is the readlines() function but this is "static". If new lines are added after I called readlines() these new lines will not be read.

How can I do this in python ? Thanks

Does this answer your question? [Reading from a frequently updated file](https://stackoverflow.com/questions/5419888/reading-from-a-frequently-updated-file) — broAhmed, Mar 23 '21 at 21:07

wallabra · Answer 1 · 2021-03-23T21:22:52.353

You will want to use a "state machine". That is just a fancy way of saying, you want to keep track of where in the file you are, so you amy seek() to that position, read() to the end of file, and read and advance the current position everytime you encounter a newline.

You could use something like this, which could also be used like an iterator:

import time


class LogFollower:
    def __init__(self, fp):
        self.position = 0
        self.fp = fp

    def seek(self):
        self.fp.seek(self.position)

    def has(self):
        self.seek()
        return '\n' in self.fp.read()

    def __iter__(self):
        while self.has():
            self.seek()
            line = self.fp.read().split('\n')[0]
            yield line

            # advance position - this is the 'state machine' part!
            self.position += len(line) + 1

follow = LogFollower(open("my_file.txt"))

# assume the file already has 2 lines

for line in follow:
    print(line)

#>foo
#>bar

time.sleep(5)

# now suppose a line 'baz' is added to the bottom
# somewhere during those 10 secs, then you decide
# to iterate again.

for line in follow:
    print(line)

#>baz

You can also continuously check it for new lines by iterating again, like shown in the hypothetical example above, when baz is appended.

Please note that, this way, every line must end with a linefeed (\n). This makes things simpler, and I suppose might be why it is the usual convention anyway.

This example takes a slightly more hands-on approach than just a simple readline loop like this one. I reckon it takes a bit more line-count this way. However, I believe it is cleaner for illustrative purposes; I suppose it does a good enough job at explaining the basics of the task with simple object-oriented programming.

P.S. I probably call seek a couple more times than I really need to. I could, for instance, not call it in the has() function, after each run of the for loop in __next__. I decided to keep it this way, though, for illustrative clarity. :)

P.P.S. I know it's not really what a state machine is. I mean in a very broad sense. Actual finite state machines are an entirely different concept. All this does is increment a counter every time it encounters a new line. I hope it's not too misleading, and that the actual point I was trying to make still remains clear – keeping track.

Continuously read the last line of log file in python

1 Answers1