0

I have a python script which is reading a continuously updated log file line by line. During some instances the function is reading a single line as three lines.

Below is the function which is reading the log file continuously.

def follow(thefile):
    print("stilllll")
    thefile.seek(0, 2)
    while True:
        #time.sleep(0.1)
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

Below code is for calling this function from main.

if __name__ == '__main__':
    logfile = codecs.open("log.txt", "r", encoding='utf-8',errors='ignore')
    loglines = follow(logfile)
    for line in loglines:

Below is how logging is happening and log.txt is the file which is being written and read.

cat my1.fifo | telnet 172.27.172.247 10031 &> log.txt &

The below is the screenshot of line which was read as three lines.

enter image description here

It read it as three lines like this

0.0
[sudo] password
0.1643835616438356
for olta
0.09230769230769231
usr:
0.06451612903225806

What could be the reason for it ?

I tried to change the way it is reading the log file but since it is happening only some times then it is difficult to reproduce.

Anubhav Rai
  • 11
  • 1
  • 4
  • It looks like race condition, you read the line while being written in separate thread/process, while the thread in which you read is sleeping – buran Apr 06 '23 at 12:44
  • If you want to have both file handler and console handler, why not setup logging with 2 handlers? i.e. this looks like XY problem to me – buran Apr 06 '23 at 12:45
  • How can we prevent it from happening ? The log is written by a telnet process . – Anubhav Rai Apr 06 '23 at 12:46
  • Maybe this helps: https://stackoverflow.com/q/12523044/4046632 – buran Apr 06 '23 at 12:49
  • The problem is the write is happening from somewhere else and it is not like both read and write are happening from the same code. So the above link did not help. – Anubhav Rai Apr 06 '23 at 13:03
  • The main takeaway from the question I linked - don't yield until `line` ends with `'\n'` – buran Apr 06 '23 at 13:47
  • yeah that can work but needs to be checked – Anubhav Rai Apr 06 '23 at 13:58

0 Answers0