Programmatically exiting python script while multithreading

Question

I have some code which runs routinely, and every now and then (like once a month) the program seems to hang somewhere and I'm not sure where.

I thought I would implement [what has turned out to be not quite] a "quick fix" of checking how long the program has been running for. I decided to use multithreading to call the function, and then while it is running, check the time.

For example:

import datetime
import threading

def myfunc():
  #Code goes here

t=threading.Thread(target=myfunc)
t.start()
d1=datetime.datetime.utcnow()
while threading.active_count()>1:
  if (datetime.datetime.utcnow()-d1).total_seconds()>60:
    print 'Exiting!'
    raise SystemExit(0)

However, this does not close the other thread (myfunc).

What is the best way to go about killing the other thread?

If you want other threads to die immediately when `SystemExit` is raised, make them daemon threads. — roippi, Sep 23 '13 at 03:46

score 3 · Accepted Answer · answered Sep 23 '13 at 03:59

The docs could be clearer about this. Raising SystemExit tells the interpreter to quit, but "normal" exit processing is still done. Part of normal exit processing is .join()-ing all active non-daemon threads. But your rogue thread never ends, so exit processing waits forever to join it.

As @roippi said, you can do

t.daemon = True

before starting it. Normal exit processing does not wait for daemon threads. Your OS should kill them then when the main process exits.

Another alternative:

import os
os._exit(13)  # whatever exit code you want goes there

That stops the interpreter "immediately", and skips all normal exit processing.

Pick your poison ;-)

Be aware that this breaks 'atexit' callbacks – Mariusz Jamro Dec 19 '14 at 09:47 — Mariusz Jamro, Dec 19 '14 at 09:47

score 1 · Answer 2 · edited May 23 '17 at 12:11

There is no way to kill a thread. You must kill the target from within the target. The best way is with a hook and a queue. It goes something like this.

import Threading
from Queue import Queue

# add a kill_hook arg to your function, kill_hook
# is a queue used to pass messages to the main thread
def myfunc(*args, **kwargs, kill_hook=None):
  #Code goes here
  # put this somewhere which is periodically checked.
  # an ideal place to check the hook is when logging
  try:
    if q.get_nowait():  # or use q.get(True, 5) to wait a longer
      print 'Exiting!'
      raise SystemExit(0)
    except Queue.empty:
      pass

q = Queue()  # the queue used to pass the kill call
t=threading.Thread(target=myfunc, args = q)
t.start()
d1=datetime.datetime.utcnow()
while threading.active_count()>1:        
  if (datetime.datetime.utcnow()-d1).total_seconds()>60:
  # if your kill criteria are met, put something in the queue
    q.put(1)

I originally found this answer somewhere online, possibly this. Hope this helps!

Another solution would be to use a separate instance of Python, and monitor the other Python thread, killing it from the system level, with psutils.

Wow, I like the daemon and stealth os._exit solutions too!

Programmatically exiting python script while multithreading

2 Answers2