7

I can do:

python script.py > logfile 2>&1

and once I kill it using ctrl c I can see changes to the logfile. However, when I do:

python script.py > logfile 2>&1 &

I can't see any changes to the log file . How can I background my script and still have it write to the logfile?

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
user1689987
  • 1,002
  • 1
  • 8
  • 23
  • 3
    Buffering. Explicitly flush. – Charles Duffy Apr 03 '15 at 15:18
  • By the way -- the typical reason this works differently between going to stdout and going to a file is that the standard C library's behavior differs based on whether the output descriptor being written to is a TTY; TTY output is line-buffered or unbuffered by default, whereas output to a file is not. (I say "typical" because the Python runtime may do its own thing rather than leaning on the C library here, but the end effect is similar). – Charles Duffy Apr 03 '15 at 15:23

1 Answers1

11

Your writes aren't going into the ether -- they're getting buffered. After a graceful shutdown (SIGTERM, not SIGKILL), you should see content in the file then as well.

Alternately, you can explicitly flush:

sys.stdout.flush()

...or tell Python to run in unbuffered mode:

python -u yourscript.py

or start your script with

#!/usr/bin/python -u

...to ensure that content is written. This will, of course, have a performance penalty, since your script will be trying to perform IO more frequently.

(As an aside, #!/usr/bin/env python -u is not guaranteed to work: Only a command name and a single argument are guaranteed to be passed through from a shebang line to a command that's run; your operating system may or may not honor more than that).

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • thanks! for non python commands I found: [how to flush](http://stackoverflow.com/questions/1429951/bash-how-to-flush-output-to-a-file-while-running) – user1689987 Apr 03 '15 at 15:23
  • The accepted answer there on that question boils down to "you need to solve it within the bounds of the specific program you're running" -- not tremendously helpful. `stdbuf` is a GNU extension, and `unbuffer` is packaged with `expect` -- not a POSIX-standardized tool. There aren't really good generic answers here. – Charles Duffy Apr 03 '15 at 15:25
  • 1
    Personally, if I wanted to point to a good generic reference, I'd make that BashFAQ #009: http://mywiki.wooledge.org/BashFAQ/009 – Charles Duffy Apr 03 '15 at 15:26