1

I'm trying to read a log file under macOS with Python2 (this combination is important, Python 3 works ok, Linux hasn't this problem with both versions).

This file is written by another program at the same time.

Issue: new lines (appeared after last reading cycle) are unavailable for Python. (but I see them with tail -f). To reproduce you can run reader and writer in the same directory in different terminals (and tail -f out.out in third terminal if you want to check):

Reader:

import os
import time

def str_num(line):
    n_s = line.find('#')
    n_e = line[n_s:].find(' ')
    print(line)
    return int(line[n_s+1: n_s + n_e])

with open('out.out', 'rt') as _file:
    while True:
        os.path.getsize('out.out')
        lines = _file.readlines(1000)
        n1 = n2 = -1
        if lines: 
            n1 = str_num(lines[0])
            n2 = str_num(lines[-1])
        print("%s (%s - %s)" % (len(lines), n1, n2))
        time.sleep(1)

writer:

import time
num = 0
with open('out.out', 'w', buffering=1)as _file:
    while True:  
        _file.write("*** String #%s ***\n" % num)
        if not num % 100: print(num)
        num += 1
        time.sleep(0.01)

One workaround I've found is to use tell() and seek() to 'reset' file access. Is there any smarter way to resolve this problem?

bfontaine
  • 18,169
  • 13
  • 73
  • 107
greyfenrir
  • 19
  • 2
  • If you see them with `tail -f` and you don't have any good solution/workaround, an option could be to use `subprocess` to run `tail -f` (or `os.popen` if you don't really care). – zneak Sep 28 '17 at 08:15
  • 1
    Thanks, but I prefer to resolve this with standard python tools. – greyfenrir Sep 28 '17 at 08:26

1 Answers1

0

I think you may want a+:

with open('out.out', 'a+') as _file:

as per this table:

                  | r   r+   w   w+   a   a+
------------------|--------------------------
read              | +   +        +        +
write             |     +    +   +    +   +
create            |          +   +    +   +
truncate          |          +   +
position at start | +   +    +   +
position at end   |                   +   +

Please see the question at: python open built-in function: difference between modes a, a+, w, w+, and r+?

neophytte
  • 648
  • 2
  • 11
  • 25
  • Have you checked your answer? It doesn't work for me. If I understand correctly you offer to open file at the end (and skip it's contents) and it's only difference with my variant. – greyfenrir Sep 28 '17 at 08:29