Python uncaught exceptions in main not killing threads

Question

When I run a python program (2.7 or 3), I import a module that has a class that initializes some threads. The problem is whenever an uncaught exception occurs in the main thread, the main function dies yet the thread keeps running like a zombie causing the python process to never die.

What's the best way to have any uncaught exception in the main thread (or even other threads) to kill everything everywhere.

Since I often call the subprocess module, I usually use threading.Event to help exit cleanly. However, uncaught exceptions won't trigger those events.

Here's an example of a program where the thread just won't die....

prog1.py

#!/usr/bin/env python2.7

import threading
import modx1

mod_obj = modx1.Moddy()

raise Exception('DIE UNEXPECTEDLY')

try:
    raise Exception('known problem here')
except Exception:
    mod_obj.kill_event.set()

modx1.py

#!/usr/bin/env python2.7

import threading
import subprocess
from time import sleep


class Moddy():
    def __init__(self):
        self.kill_event = threading.Event()
        self.my_thread=threading.Thread(target=self.thread_func)
        self.my_thread.start()
    def thread_func(self):
        while  not self.kill_event.is_set():
            print('thread still going....')
            sleep(2)

Answer 1

This seems to be hackish, but still something to start with.

prog1.py - I have added sleep of 5 seconds so you can see the thread running and then gets completed.

import modx1
import time


mod_obj = modx1.Moddy()
time.sleep(5)
raise Exception('DIE UNEXPECTEDLY')

try:
    raise Exception('known problem here')
except Exception:
    mod_obj.kill_event.set()

modx1.py -> Check for the boolean you set and also the status of main thread.

import ctypes
import threading
import subprocess
from time import sleep


class Moddy():
    def __init__(self):
        self.kill_event = threading.Event()
        self.my_thread = threading.Thread(target=self.thread_func)
        self.my_thread.start()

    def thread_func(self):
        while not self.kill_event.is_set() and threading.main_thread().is_alive():
            print('thread still going....')
            sleep(2)

OUTPUT

thread still going....
thread still going....
thread still going....
Traceback (most recent call last):
  File "/Users/gaurishankarbadola/PycharmProjects/untitled1/prog1.py", line 8, in <module>
    raise Exception('DIE UNEXPECTEDLY')
Exception: DIE UNEXPECTEDLY

Process finished with exit code 1

Answer 2

Assuming you're not reliant on clean up occurring in the threads, the simplest solution is to just launch all your threads as daemon threads; a Python program exits when all non-daemon threads have completed, so if your main thread is the only non-daemon thread, the program will terminate when the main thread does.

The change required is trival, just change:

self.my_thread=threading.Thread(target=self.thread_func)

to:

self.my_thread=threading.Thread(target=self.thread_func, daemon=True)

and the thread will be launched as a daemon.

If you do have cleanup that must occur and isn't automatic on program termination (e.g. sending some "I'm done" message to a remote machine over the network, where simply closing the connection isn't sufficient for whatever reason), daemon won't work (the threads are forcibly terminated, whatever they were in the middle of), so you can use your approach, you just need to expand the try block to cover all code from the moment the thread is launched so the handling always occurs. I'd also recommend set-ing the Event in a finally block, not an except block, if you want to always tell the threads to clean up, not just when something bad happens. If it should be done only on an exception, I'd use a bare except: block, and explicitly re-raise after setting Event, e.g.:

try:
    # Entire program after thread launches
except:
    mod_obj.kill_event.set()
    raise  # Reraises exception as if it were never caught, rather than silencing it