Writing to file from a python script started with rc.local is missing the first write using python and linux

Question

Im using a raspberry pi with raspbian, Debain Wheezy Jan 2014 and python3

I'm starting a python script from rc.local that captures a keyboard input and writes to a file, without logging in.

If the file that the script is writing to has not been created yet, the first keyboard input registers on the screen but isn't written to the file. All subsequent writes work fine.

My code works fine when I run it from the command line as user that's logged in, the first line is written to the new file as expected.

EDITED CODE FROM MIDNIGHTER

#!/usr/bin/env python3.2
import sys
from datetime import datetime

def main():
    f = open('/home/pi/cards.csv','r')
    sim = f.read()
    sim = sim.split('\n')
    simSet = set(sim)

    while True:
        try:
            log = open('logs', 'a')
            puk = input() # text input, i.e., always a string
            included = "true" if puk in simSet else "false"
            print(included, puk)
            log.write("{included: %s, time: %s, number: %s}, \n" % (included, datetime.now(), puk))
            log.close()
        except ValueError:
            log.close()

main()

And the rc.local

 sudo python3 /home/pi/rf1

Im just learning this, please excuse the poor execution.

SOLUTION

I realise now I left out an important detail about a cron job closing and copying the file that was being written to.

I found my answer here what exactly the python's file.flush() is doing?

Instead of file.close.() I used file.flush() and it works.

Code below:

#!/usr/bin/env python3.2
import sys
from datetime import datetime

def main():
    f = open('/home/pi/cards.csv','r')
    sim = f.read()
    sim = sim.split('\n')
    simSet = set(sim)
    log = open('logs', 'a')
    while True:
        try:
            puk = input() # text input, i.e., always a string
            included = "true" if puk in simSet else "false"
            print(included, puk)
            log.write("{included: %s, time: %s, number: %s}, \n" % (included, datetime.now(), puk))
            log.flush()
        except ValueError:
            log.flush()

main()

is `phoneNumbers` meant to be `sims`? Can't see `phoneNumbers` being defined anywhere. — elParaguayo, May 13 '14 at 11:37
Some comments on your code in general: 1. You should turn sims into a `set` so that the test `puk in sims` has constant complexity. 2. I'm not sure what will raise a ValueError in your code. 3. You can aggregate much of the code in the while loop: `puk = input() # text input, i.e., always a string included = "true" if puk in sims else "false" print(included, puk) log.write(...)` If it doesn't cause trouble, I would move opening of the file outside the loop and use a `with` statement. 4. If your code does actually raise a ValueError you should close your log in a finally clause. — Midnighter, May 13 '14 at 12:05
Thanks Midnighter, I needed to learn that at some stage. Ive made the changes and updated the code, still having the same problem.. I cant move the open file function outside the loop as it causes problems. — fairywings78, May 14 '14 at 11:03
If you have a solution, you should post it as an answer to your own question. — deadly, May 14 '14 at 12:56

score 0 · Accepted Answer · edited May 23 '17 at 12:27

The problem was I was running a cron job that copied the data to another file which was accessing the file being written to in the python program.

The first write after this was not saving to the file as it was being accessed by another program.

These paragraphs seem to be what was happening:

https://stackoverflow.com/a/7127162/1441620

The first, flush, will simply write out any data that lingers in a program buffer to the actual file. Typically this means that the data will be copied from the program buffer to the operating system buffer.

Specifically what this means is that if another process has that same file open for reading, it will be able to access the data you just flushed to the file. However, it does not necessarily mean it has been "permanently" stored on disk.

I think @Midnighter also suggested using a withstatement to open and close the file would have also solved it.

Updated code is in the question > solution.

Writing to file from a python script started with rc.local is missing the first write using python and linux

1 Answers1