Clean shutdown of a Python script

Question

I have a threaded server written in Python that I start using the following shell script:

#!/bin/bash

base_path="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

public_dns=$(curl -s http://169.254.169.254/latest/meta-data/public-hostname)
echo $public_dns > "$base_path/client/address"

cd "$base_path/server"
python "server.py" &
echo $! > "$base_path/server_pid"
echo "Server running"

I echo the PID to a file so that I can shutdown a server using another shell script:

#!/bin/bash

base_path="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

kill -9 `cat "$base_path/server_pid"`
rm "$base_path/server_pid"
rm "$base_path/client/address"

I know, however, that this is a bad approach considering the server has many threads that have I/O into network and hdd... So what I would like to do is have the second script somehow interact with the server and tell it to start a shutdown sequence which would cleanly close all the threads, close & archive logs etc.

Now I know about atexit and I tested it this way:

import atexit
def byebye(o):
    o.write('stop')
    o.flush()
    o.close()

o = open('log','w')
o.write('start')
o.flush()

atexit.register(byebye, o)

while True:
    pass

But when I kill -9 the process, byebye() is not fired. Should I use a command other than the almighty kill -9? How would I go about shutting down the process?

What about sockets? You could either write to a local file socket or use a network socket (specific port maybe and only communicated via localhost). There you could tell it to shut down. Kill seems to be a bit harsh... — javex, Jul 18 '12 at 15:13
As long as you don't have any `try/except KeyboardInterrupt` clauses, you could use `kill -2` as that will send SIGINT. A more clean solution would be to register a signal handler in python which call `sys.exit` -- Although I'm not sure how that would work since you're in a threaded environment... — mgilson, Jul 18 '12 at 15:17
I have thought about just writing the word 'stop' to a file which would be constantly monitored by the server, but thats another thread to launch and I thought that maybe there is a more elegant way to do this in Python — Dreen, Jul 18 '12 at 15:17
The `cd` is unnecessary; `base_path=$(dirname "${BASH_SOURCE[0]}")` — chepner, Jul 18 '12 at 15:45
@mgilson: after some tests I think your approach is best, post it as answer so that I may accept it. — Dreen, Jul 20 '12 at 04:29

score 5 · Answer 1 · edited May 23 '17 at 12:27

5

It seems to me that your are trying to implement a daemon. There is various references about daemon implementations in python :

Previous stackoverlow question : How do you create a daemon in Python?

PEP regarding daemon : http://www.python.org/dev/peps/pep-3143/

python module implementing PEP3143 : http://pypi.python.org/pypi/python-daemon/

Note that the PEP is a draft.

More about daemons : http://en.wikipedia.org/wiki/Daemon_%28computing%29

Typically, daemon are started, stopped and restarted as :

mydaemon start
mydaemon stop
mydaemon restart

So you dont need to know the PID or anything else to stop it.

edited May 23 '17 at 12:27

Community

1
1

answered Jul 18 '12 at 15:17

Nicolas Barbey

6,639
4
28
34

The server is based on the SocketServer (http://docs.python.org/library/socketserver.html) which already runs as a daemon (by `server_thread.daemon = True`), so wrapping my script to be a daemon which starts a daemon seems a bit unnecessary? – Dreen Jul 18 '12 at 17:17
I am not an expert in daemon and networking application but it seems that the functionality provided by this server_thread.daemon option is not the same as the one provided by the python-daemon module. In particular, the convenient command-line interface provided by python-daemon is not included in the SocketServer module. – Nicolas Barbey Jul 19 '12 at 11:02

score 1 · Answer 2 · answered Jul 18 '12 at 15:21

kill 9 is atomic powered - you don't get to clean up after yourself. A better approach would be to use a different, gentler, signal (HUP is commonly used to signal a server process that it is time to shut down, for example), and teach your python code how to handle it gracefully.

The signal module documentation should get you started.

score 1 · Accepted Answer · answered Jul 20 '12 at 12:11

I'm not particularly used to programming with threads, but rather than sending kill -9 (which corresponds to SIGKILL), you could send SIGINT or some other user defined signal. SIGINT (kill -2 on my system) is nice because that one is already understood by python. (When python catches that signal, it raises a KeyboardInterrupt), but any signal will work. You just need to register a signal handler that exits your program cleanly.

Clean shutdown of a Python script

3 Answers3

Linked