182

I am using the following command to run a python script in the background:

nohup ./cmd.py > cmd.log &

But it appears that nohup is not writing anything to the log file. cmd.log is created but is always empty. In the python script, I am using sys.stdout.write instead of print to print to standard output. Am I doing anything wrong?

  • Which variant of `nohup` are you using? The BSD version writes to a file called `nohup.out` in the current directory (or `$HOME/nohup.out` if the current directory isn't writable). I don't see a way to change the output file name... – wulong Oct 16 '12 at 17:10
  • @wulong That's only if if stdout is a terminal. – John Kugelman Oct 16 '12 at 17:11
  • I also tried the command without redirection and it didn't create the nohup.out file at all. I don't know which variant it is but I am on SunOS 5.10 if that helps. –  Oct 16 '12 at 17:17

7 Answers7

513

You can run Python with the -u flag to avoid output buffering:

nohup python -u ./cmd.py > cmd.log &
vz0
  • 32,345
  • 7
  • 44
  • 77
  • @kommradHomer I guess it depends on the amount of output on stdout/stderr your program produces. – vz0 Dec 10 '14 at 09:58
  • 1
    Warning: it [doesn't always work](http://stackoverflow.com/q/34753765/1422096). I don't know why. Do you? – Basj Jan 12 '16 at 21:41
115

It looks like you need to flush stdout periodically (e.g. sys.stdout.flush()). In my testing Python doesn't automatically do this even with print until the program exits.

wulong
  • 2,657
  • 1
  • 20
  • 19
  • 22
    python as well as other C stdio-based programs uses line-buffering in interactive case (stdout is connected to a tty) and block-buffering when redirected to a file. If `python -u` doesn't work; `nohup` might have introduced its own buffering. – jfs Oct 16 '12 at 17:37
  • 16
    @J.F.Sebastian As of today, `nohup` doesn't buffer output and `python -u` works just fine. (just an update for people) – Pijusn Nov 19 '14 at 08:31
  • 1
    @Pius: [`nohup` is a POSIX utility](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/nohup.html) there could be different implementations on different platforms. btw, python3 I/O is no longer C stdio-based but it has similar buffering behavior. – jfs Dec 01 '14 at 16:30
  • @jfs stderr buffering behavior changed in Python 3.9, it is now always line-buffered. See [here](https://bugs.python.org/issue13601). – jdhao Mar 16 '21 at 17:07
76

  • Using -u with nohup worked for me. Using -u will force the stdout, stderr streams to be unbuffered. It will not affect stdin. Everything will be saved in "nohup.out " file. Like this-

    nohup python -u your_code.py &

    You can also save it into your directory. This way- 

    nohup python -u your_code.py > your_directory/nohup.out &

  • Also, you can use PYTHONUNBUFFERED. If you set it to a non-empty string it will work same as the -u option. For using this run below commands before running python code. 

    export PYTHONUNBUFFERED=1

    or

    export PYTHONUNBUFFERED=TRUE

P.S.- I will suggest using tools like cron-job to run things in the background and scheduled execution.

Nurul Akter Towhid
  • 3,046
  • 2
  • 33
  • 35
21
export PYTHONUNBUFFERED=1
nohup ./cmd.py > cmd.log &

or

nohup python -u ./cmd.py > cmd.log &

https://docs.python.org/2/using/cmdline.html#cmdoption-u

Ryu_hayabusa
  • 3,666
  • 2
  • 29
  • 32
5

Python 3.3 and above has a flush argument to print and this is the only method that worked for me.

print("number to train = " + str(num_train), flush=True)
print("Using {} evaluation batches".format(num_evals), flush=True)
Ganesh Krishnan
  • 7,155
  • 2
  • 44
  • 52
0

I had a similar issue, but not connected with a Python process. I was running a script which did a nohup and the script ran periodically via cron.

I was able to resolve the problem by:

  1. redirecting the stdin , stdout and stderr
  2. ensuring the the script being invoked via nohup didn't run anything else in the background

PS: my scripts were written in ksh running on RHEL

Pradeep Anchan
  • 1,469
  • 1
  • 11
  • 11
-1

I run my scripts in the following way and I have no problem at all:

nohup python my_script.py &> my_script.out &

comparing with your syntax looks like you are only missing a "&" symbol after your input...

teff ros
  • 29
  • 1
  • 6