Wait for text file to be written, read changes line by line and print

Question

I have the below code, which waits for a log file to appear, it's being saved there by an external programme, when the log file appears I open it, while it's still being written to, and print the contents, as the log file is being updated by the external programme I want to print out any new lines being written to it. Currently I clear the print out and re-print the entire text, which I would like to avoid, I would like to only print the new lines. Another thing to improve is waiting for the file to appear, rather than just pausing the python script.

    a=0
    while a <= 10:
        if os.path.isfile(logPath):
            infile = file(logPath)
            break
        a=a+1
        time.sleep(1)
    fileSize1 = 0
    while True:
        wx.Yield()
        fileSize = os.path.getsize(logPath)
        if fileSize > fileSize1:
            lines = infile.readlines()
            log.Clear()
            log.Refresh()
            for line in lines:
                print line.rstrip()
            if "DBG-X: Returning 1" in line:
                break
            if "DBG-X: Returning 0" in line:
                break
        fileSize1 = fileSize
        infile.seek(0)

Keep a `LastLinePrinted` variable and check if the number of lines in the file is greater? — Basic, Feb 12 '14 at 16:37
Sounds good, how do I check if the number of lines is greater? — speedyrazor, Feb 12 '14 at 16:39
Have a look at this answer... http://stackoverflow.com/a/850962/156755. It profiles the different ways of counting lines in a file. — Basic, Feb 12 '14 at 16:41
This question has answers here you may find interesting (specially the link from first answer): [Reading from a frequently updated file](http://stackoverflow.com/questions/5419888/reading-from-a-frequently-updated-file) — Cilyan, Feb 12 '14 at 16:59
Sorry, i'm still not getting it, could someone give a working example please? — speedyrazor, Feb 12 '14 at 21:33

score 0 · Accepted Answer · answered Feb 12 '14 at 22:22

0

Try something like this...

lastSize = 0
lastLineIndex = 0
while True:
    wx.Yield()
    fileSize = os.path.getsize(logPath)
    if fileSize > lastSize:
        lines = infile.readlines()
        newLines = 0
        for line in lines[lastLineIndex:]:
            newLines += 1
            print line.rstrip()
        lastLineIndex += newLines

        if "DBG-X: Returning 1" in line:
            break
        if "DBG-X: Returning 0" in line:
            break
    fileSize1 = fileSize
    infile.seek(0)

The key bit is

for line in lines[lastLineIndex:]:

Where we skip the lines we've already seen. You can probably skip the newLines variable and just do lastLineIndex += 1 but I've got an aversion to changing lastLineIndex inside the for loop (a habit I've picked up to avoid issues in other languages)

answered Feb 12 '14 at 22:22

Basic

26,321
24
115
201

Cheers Basic. Thats problem 1 solved. Any ideas about waiting for the file to appear, rather than just pausing the python script? – speedyrazor Feb 13 '14 at 05:24
Try this answer... http://stackoverflow.com/a/597962/156755 You can monitor file system events (like file created) and check the filename... – Basic Feb 13 '14 at 09:11
This all works well. When running CPU gets maxed out, as the process runns for about 45 mins, is there a way yo dump / flush memory CPU each loop, as I might then immediately just run another after, so flushing in-between might be a good idea., but how would i do this? – speedyrazor Feb 13 '14 at 14:59
@speedyrazor Please ask a new question - it will help others in future and avoid unrelated clutter. – Basic Feb 13 '14 at 19:25

Wait for text file to be written, read changes line by line and print

1 Answers1