Starting/stopping a background Python process wtihout nohup + ps aux grep + kill

Question

I usually use:

nohup python -u myscript.py &> ./mylog.log &       # or should I use nohup 2>&1 ? I never remember

to start a background Python process that I'd like to continue running even if I log out, and:

ps aux |grep python
# check for the relevant PID
kill <relevantPID>

It works but it's a annoying to do all these steps.

I've read some methods in which you need to save the PID in some file, but that's even more hassle.

Is there a clean method to easily start / stop a Python script? like:

startpy myscript.py             # will automatically continue running in
                                # background even if I log out

# two days later, even if I logged out / logged in again the meantime
stoppy myscript.py

Or could this long part nohup python -u myscript.py &> ./mylog.log & be written in the shebang of the script, such that I could start the script easily with ./myscript.py instead of writing the long nohup line?

Note : I'm looking for a one or two line solution, I don't want to have to write a dedicated systemd service for this operation.

Well, you can use [`os.system()`](https://docs.python.org/3/library/os.html#os.system) or [`subprocess`](https://docs.python.org/3/library/subprocess.html#subprocess) to run a program in the backgound. You can also use [`os.kill()`](https://docs.python.org/3/library/os.html#os.kill) instead of GNU `kill`. [And here's some ways to get a list of running processes.](http://stackoverflow.com/questions/2703640/process-list-on-linux-via-python) But something I don't understand: What do you mean about *two days later, even after some reboot* ? — Remi Guan, Jan 08 '16 at 23:48
@KevinGuan : Sorry, I edited my question (I meant: *two days later even after some log out / log in*). About the first part of your comment: no, I don't want to start or kill a process *from* Python (with `os` module). I want to start / stop a background-running-Python process easily, in one or two lines, from bash. — Basj, Jan 08 '16 at 23:54
Ah, that's fine. I have removed Python tag and added Bash tag since actually you're not *programming in Python*. — Remi Guan, Jan 08 '16 at 23:57
By the way, are you looking for `pkill` or `killall` command? Which kills a process by it's name. — Remi Guan, Jan 08 '16 at 23:58
@KevinGuan Ok for adding bash tag, but I have set python tag again, because sometimes there are specific behaviour when using Python (for example `nohup python myscript...` works bad whereas `nohup python -u myscript...` ensures everyting is written, unbuffered etc.) — Basj, Jan 08 '16 at 23:59
`pkill` or `killall` seem to be what you are looking for. Another very hacky, but simple method, is to check in the main loop of your Python for the existence of a file called `/STOP` and delete it and exit if it exists, then you can stop the script with `touch /STOP`. Likewise, you can do `touch /PAUSE` and the script can stop processing and sleep in a loop checking each time it wakes if `/PAUSE` exists. Crude but easy. — Mark Setchell, Jan 09 '16 at 08:10

score 4 · Accepted Answer · edited May 23 '17 at 11:46

As far as I know, there are just two (or maybe three or maybe four?) solutions to the problem of running background scripts on remote systems.

1) nohup

nohup python -u myscript.py > ./mylog.log  2>&1 &

1 bis) disown

Same as above, slightly different because it actually remove the program to the shell job lists, preventing the SIGHUP to be sent.

2) screen (or tmux as suggested by neared)

Here you will find a starting point for screen.

See this post for a great explanation of how background processes works. Another related post.

3) Bash

Another solution is to write two bash functions that do the job:

mynohup () {
    [[ "$1" = "" ]] && echo "usage: mynohup python_script" && return 0
    nohup python -u "$1" > "${1%.*}.log" 2>&1 < /dev/null &
}

mykill() {
    ps -ef | grep "$1" | grep -v grep | awk '{print $2}' | xargs kill
    echo "process "$1" killed"
}

Just put the above functions in your ~/.bashrc or ~/.bash_profile and use them as normal bash commands.

Now you can do exactly what you told:

mynohup myscript.py             # will automatically continue running in
                                # background even if I log out

# two days later, even if I logged out / logged in again the meantime
mykill myscript.py

4) Daemon

This daemon module is very useful:

python myscript.py start

python myscript.py stop

Is it possible to make such a bash function `myrun` that will be used with `myrun script.py` that 1) reads the first line of `script.py`, remove the leading `#MYRUN:` 2) execute the command written in this first line. Example: doing `myrun script.py` with [this file](http://pastebin.com/u7KVhMLx) will execute `nohup python -u script.py &`. — Basj, Jan 16 '16 at 16:05
I started a question about this here http://stackoverflow.com/questions/34829145/a-bash-function-that-runs-script — Basj, Jan 16 '16 at 16:19
I now use `screen` and that's awesome and simple. `screen -s myscript`, then run the script `python myscript.py` then detach with CTRL+A, D, then I can log out etc. To reattach `screen -r myscript`, that's it! — Basj, Dec 07 '16 at 12:17

score 1 · Answer 2 · answered Jan 09 '16 at 00:13

1

Do you mean log in and out remotely (e.g. via SSH)? If so, a simple solution is to install tmux (terminal multiplexer). It creates a server for terminals that run underneath it as clients. You open up tmux with tmux, type in your command, type in CONTROL+B+D to 'detach' from tmux, and then type exit at the main terminal to log out. When you log back in, tmux and the processes running in it will still be running.

answered Jan 09 '16 at 00:13

neallred

740
1
8
24

Yes I mean log in and out remotely from SSH (the place where the script is run is a distant server). What happens to tmux when I log out from SSH? Will the Python script continue? Something else: how to avoid to write `nohup python -u myscript.py &> ./mylog.log &` every time? Can I put this thing in shebang? – Basj Jan 09 '16 at 09:13
When you logout with `logout` or with `exit`, the script will continue to run either way. The only things you will need to type are `tmux`, `command to run`, `CTRL + B + D` (to detach from tmux), and `logout`. When you log back in and want to stop the script, tmux will still be running. You can connect to it with `tmux attach`. As another plus, you can run as many terminal clients in one window as you want. See here for its many configuration options: http://manpages.ubuntu.com/manpages/precise/en/man1/tmux.1.html – neallred Jan 09 '16 at 15:47

Starting/stopping a background Python process wtihout nohup + ps aux grep + kill

2 Answers2