Get only new lines from file

Question

Currently i have this piece of code, but it reads all lines, and then watch the file with the while True statement:

with open('/var/log/logfile.log') as f:
        while True:
            line = f.readline()
            if not line:
                time.sleep(1)
            else:
                print(line)

I actually only need new lines coming after the lines already detected when opening the file - anybody who can help me out? Maybe a better way to watch it aswell, than with the while statement?

Another problem is that on Linux machines the script is actually locking the file so it can not be written to, before i close the script again. On OS X it works fine. Could also be nice with an idea to work around this.

Hope there is somebody out there who have been working with something similar.

score 5 · Accepted Answer · edited May 23 '17 at 12:31

5

You could initially read the file completely, then close it, and keep a change monitor over it, this monitor is implemented below using polling.

import time

filePath = '/var/log/logfile.log'

lastLine = None
with open(filePath,'r') as f:
        while True:
            line = f.readline()
            if not line:
                break
            print(line)
            lastLine = line

while True:
    with open(filePath,'r') as f:
        lines = f.readlines()
    if lines[-1] != lastLine:
        lastLine = lines[-1]
        print(lines[-1])
    time.sleep(1)

But you could also use tools like those described at: Detect File Change Without Polling

edited May 23 '17 at 12:31

Community

1
1

answered Jul 18 '14 at 06:35

Pablo Francisco Pérez Hidalgo

27,044
8
36
62

This one is great, but still locking the file in Linux - any idea why? – Adionditsak Jul 18 '14 at 06:59
@Adionditsak Try removing `with` sentence in the second loop, just: `f = open(...)` or getting out the lines from `if lines[-1]...` from the `with` block – Pablo Francisco Pérez Hidalgo Jul 18 '14 at 07:03

score 2 · Answer 2 · 2014-07-18T07:41:22.747

2

with open('/var/log/logfile.log','r') as f:
    for line in f:
        print line

edited Jul 18 '14 at 07:41

answered Jul 18 '14 at 06:36

1

Your last line `f.close()` is unnecessary since `f` will be closed at the end of the `with` block. – Pablo Francisco Pérez Hidalgo Jul 18 '14 at 07:12
@PabloFranciscoPérezHidalgo We need to close the file **f** explicitely otherwise troubles will occur depending on the nature and the scope of the application. I give you just a quick and short example: try to append data with more than one line (open file with `a` and write: yourfile.write("line 1") ... line 2 ... line 3. Don note type `yourfile.close()`. check your file with `cat yourfile.txt` (for example) and see if your lines are inserted or not. – Jul 18 '14 at 07:18
2

As far as I understand, the resource opened within `with` will be closed at the end of its block. Answers like these: http://stackoverflow.com/questions/1369526/what-is-the-python-keyword-with-used-for seem to reinforce that idea. I have performed the test you suggested and I got the expected result, with all the lines inserted. Could you post an example where the situation you describe occurs? – Pablo Francisco Pérez Hidalgo Jul 18 '14 at 07:27
1

@PabloFranciscoPérezHidalgo I followed the [link](https://docs.python.org/2/whatsnew/2.6.html#pep-343-the-with-statement) on the accepted answer of the question you showed to me. The official documentation says what you told me. I went back to a small script I did few days ago: I guessed that my problem had nothing to do with closing the file. I apologize for giving a wrong information. I updated my answer. Thank you very much for correcting me. – Jul 18 '14 at 07:41

score 0 · Answer 3 · answered Jul 18 '14 at 06:35

0

Try This,

readfile = open('/var/log/logfile.log','r')
if readfile:
   for lines in readfile:
      print lines
readfile.close()

answered Jul 18 '14 at 06:35

user3136030

384
4
9
19

Get only new lines from file

3 Answers3

Linked