2

I have a long-running python function, executing as part of a http request handler (it's tied to various options selected on a form).

The http request handler waits for the function to complete execution, and then serves an appropriate response back to the originator.

However, since the function can take some time to execute, the request may, in that time, be interrupted - causing the request handler to die, and interrupting the execution of the long-running function. This is undesirable - I'd rather make sure that the function completes execution every time it is called.

I've played with using subprocesses, but they seem to die just as soon as the request handler does.

Can I/how do I fire the function in a separated thread, or process, indicating that it should be allowed to outlive its parent?

daemons seem to maybe partially solve my problem - they can outlive their parent? - but i'd still like the parent process to monitor/access the results of the child process.

That is:

  • The parent process may die before the child one does. I'd like the child process to complete in this case
  • As long as the parent process is still alive, i'd like for it to have access to the child process, to monitor it etc... If the parent process doesn't die, I'd like for it to have access to the results of the processing the child was doing.
blueberryfields
  • 45,910
  • 28
  • 89
  • 168
  • You can probably find some hints in this threat: http://stackoverflow.com/questions/6011235/run-a-program-from-python-and-have-it-continue-to-run-after-the-script-is-kille – leeladam Sep 19 '13 at 17:40
  • 1
    Not enough information. What is the interruption? An OS signal from (e.g.) Apache to a process group? An app exception specific to some python HTTP framework, possibly naively inherited across a fork? What? – pilcrow Sep 19 '13 at 20:52

1 Answers1

1

If you start the handlers as new threads, you can run them as daemons which leaves them running even if the main thread exists. The main thread is not a daemon thread so this property needs to be set for it's children explicitly before starting them.

From the manual:

A thread can be flagged as a “daemon thread”. The significance of this flag is that the entire Python program exits when only daemon threads are left. The initial value is inherited from the creating thread. The flag can be set through the daemon property.

Note: Daemon threads are abruptly stopped at shutdown. Their resources (such as open files, database transactions, etc.) may not be released properly. If you want your threads to stop gracefully, make them non-daemonic and use a suitable signalling mechanism such as an Event.

Edu
  • 2,017
  • 17
  • 23
  • Daemon threads do not get to live longer than normal threads. When all non-daemon threads have finished running, daemon threads are **abruptly terminated**. Daemon threads are the opposite of what we want here. – user2357112 Jul 27 '17 at 21:48