Where is the secret python logfile?

Question

Using the code shown below I am recording time vs. voltage readings to a logfile. I notice that if I "open" the logfile while the program is running it does not show any new data. It only shows data that was already there in logfile.txt from the previous running of the program. In order for the new data to show up in logfile.txt, I have to stop the running program(cntl-C), then reopen logfile.txt. The new data must be recorded somewhere. Why doesn't live data show up in the logfile?

    import sys, time, signal

    with open('logfile.txt', 'a') as f:
        while True:
            volts = adc.read(256, 8)
            print >> f, time(), volts

Well, it's buffered, so either use `open('logfile.txt', 'a', 0)` to open the file in unbuffered mode, or flush the file (`f.flush()`) every time you want to read the 'live' data from the file from somewhere else. — mvdwerve, Sep 14 '15 at 22:59
What program are you using to open the file? Is it meant to update when the file is updated? How much data is being written? Python's logger will flush every log event. but the OS may buffer the file streams. — Peter Wood, Sep 14 '15 at 23:01
I open the logfile manually by using the mouse and just clicking on its icon after locating if in the file manager. — Rico, Sep 14 '15 at 23:05
@Rico I meant what program is displaying the contents of the file. Does it automatically reload the file when its contents change? — Peter Wood, Sep 15 '15 at 10:23
I've been using numpy.loadtxt and collections.Counter. I think the addition of 0 to the open('logfile.txt', 'a') solved my problem. I was unaware of the unbuffered mode, I'm simply not too experienced with working with files. — Rico, Sep 16 '15 at 00:44

score 2 · Accepted Answer · answered Sep 14 '15 at 23:01

Output gets stored in a buffer in memory, and doesn't get written to disk until the buffer is full, or you close the file, or you specifically tell it to do so. You don't see new output until the buffer is flushed from RAM.

Look up the File.flush() method; I think this is what you need.

Where is the secret python logfile?

1 Answers1