Reading infinite stream - tail

Question

Problem:

Program to read the lines from infinite stream starting from its end of file.

#Solution:

import time
def tail(theFile):
    theFile.seek(0,2)   # Go to the end of the file
    while True:
        line = theFile.readline()
        if not line:
            time.sleep(10)    # Sleep briefly for 10sec
            continue
        yield line

if __name__ == '__main__':
    fd = open('./file', 'r+')
    for line in tail(fd):
        print(line)

readline() is a non-blocking read, with if check for every line.

Question:

It does not make sense for my program running to wait infinitely, after the process writing to file has close()

1) What would be the EAFP approach for this code, to avoid if?

2) Can generator function return back on file close?

_"What would be the EAFP approach for this code, without if check?"_ - Nothing really. The reason you're using an if statement is not to keep Python from raising an error, but to verify that `line` actually has useful contents to return. Python doesn't care whether or not that is true. It would yield `line` regardless without raising an error. — Christian Dean, Jul 04 '17 at 01:00
@ChristianDean EAFP does not encourage condition check of a name before referring(`yield line`) that name. — overexchange, Jul 04 '17 at 02:21
I understand that. But there's nothing to ask forgiveness _for_. Your not doing anything wrong from Python's perspective. Again, **Python does not care if** `line` **is empty or not**. It will execute `yield line` regardless. Using a `try/except` block here is practically useless. Just stick with `if`. — Christian Dean, Jul 04 '17 at 02:38
@ChristianDean That forgiveness can be to `continue` in the loop. Forgiveness section does not need to raise exception. Shouldn't I write `continue` in `except` block? — overexchange, Jul 04 '17 at 02:40
OK. But that still doesn't solve the problem. **There is not exception to ask forgiveness _for_**. No error will be raised in your function because `line` is empty. Thus, it makes no sense to ask for forgiveness. — Christian Dean, Jul 04 '17 at 02:43
@ChristianDean Ok, Yes. There is not exception to ask forgiveness. Got u — overexchange, Jul 04 '17 at 02:48
@ChristianDean How to terminate my program, if multiple data is being retrieved from multiple sources? as shown [here](https://pastebin.com/kuNBB92S) — overexchange, Jul 04 '17 at 05:16
Sorry @overexchange. I am a busy right now and cannot answer your question. If you have another question though, you should really post a new question. — Christian Dean, Jul 04 '17 at 05:20

cs95 · Accepted Answer · 2017-07-04T02:13:56.320

3

@ChristianDean in his comment answered your first question quite nicely, so I'll answer your second.

I do believe it is possible - you can use theFile's closed attribute and raise a StopIteration exception if the file was closed. Like this:

def tail(theFile):
    theFile.seek(0, 2) 
    while True:
        if theFile.closed:
            raise StopIteration

        line = theFile.readline()
        ...
        yield line

Your loop shall cease when the file is closed and the exception is raised.

A more succinct method (thanks, Christian Dean) that does not involve an explicit exception is to test the file pointer in the loop header.

def tail(theFile):
    theFile.seek(0, 2) 
    while not theFile.closed:
        line = theFile.readline()
        ...
        yield line

edited Jul 04 '17 at 02:13

answered Jul 04 '17 at 01:23

cs95

379,657
97
704
746

1

Your right, he could implement a generator this way. However, the `StopIteration` is really only superfluous. All he has to do is break from his while loop when the file closes(`while not theFile.closed`), and that's all. Python will handle raising the `StopIteration` for the generator automatically. You don't have to raise `StopIteration` unless your creating a custom generator object. – Christian Dean Jul 04 '17 at 02:10
@ChristianDean Totally makes sense! Thanks a bunch. – cs95 Jul 04 '17 at 02:14
@ChristianDean `theFile.closed`? Does this work? My program has opened `file` for reading(thru `fd`), How can `theFile.closed` be `True`? This does not work – overexchange Jul 04 '17 at 02:55
@overexchange You are familiar with passing references, yes? `tail` will refer to the same `fd`, so that when you close it outside, the attribute will update. – cs95 Jul 04 '17 at 02:56
So, this `close()` has to be done asynchronously by calling program that passed `fd` – overexchange Jul 04 '17 at 02:58
@overexchange asynchronously or synchronously... doesn't matter... but `tail` will notice the change. – cs95 Jul 04 '17 at 02:59
1

@overexchange If you liked to see that coldspeed's code does indeed work, I created a little repl.it session [here](https://repl.it/JM21/1). As you may can tell, when we call `fd.close()` inside of the for loop, the generator ends on its next call and stops short before reading all of the data in `file.txt`. This shows that it does respond to `fd` being closed from outside the function. The only thing to note is that I removed `theFile.seek(0, 2)` because that would note produce any visual output, which I thought was important to demonstrate the point. – Christian Dean Jul 04 '17 at 03:07
@ChristianDean Beauty :) – cs95 Jul 04 '17 at 03:08
@cᴏʟᴅsᴘᴇᴇᴅ It sure is. _sniff_. It sure is. – Christian Dean Jul 04 '17 at 03:09
1

@cᴏʟᴅsᴘᴇᴇᴅ Thanks. Took long enough! :-) [Now to see what new privileges](https://stackoverflow.com/help/privileges) I have ;-) – Christian Dean Jul 04 '17 at 03:14
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/148265/discussion-between-christian-dean-and-cs). – Christian Dean Jul 04 '17 at 03:26

Reading infinite stream - tail

Problem:

1 Answers1

Linked