Can not catch exception in threaded program

Question

The Python program below starts one thread and then continues to perform actions in the main thread. I wrap the whole main thread in a try-except block so I can tear down all running threads if an exception occured.

When I run the script using Python 2.7.5 and invoke a KeyboardInterrupt at a random point during the programs execution, the exception is triggered but not catched. The program continues to run.

$ python test.py 
Running server ...
Searching for servers ...
^CTraceback (most recent call last):
  File "test.py", line 50, in <module>
    main()
  File "test.py", line 40, in main
    app_main()
  File "test.py", line 35, in app_main
    searchservers()
  File "test.py", line 26, in searchservers
    time.sleep(0.0005)
KeyboardInterrupt

I miss a line in the output which is printed in main() when an exception occured.

---

Code

import time
import threading

thread_pool = []
running = False

def stop():
    global running
    running = False

def runserver():
    print "Running server ..."

    global running
    running = True

    while running:
        time.sleep(0.07)

def searchservers():
    print "Searching for servers ..."

    for i in xrange(256):
        for j in xrange(256):
            time.sleep(0.0005)

def app_main():
    server = threading.Thread(target=runserver)
    thread_pool.append(server)
    server.start()

    time.sleep(0.1)

    searchservers()
    stop()

def main():
    try:
        app_main()
    except Exception as exc:
        stop()
        print "%s occured, joining all threads..." % exc.__class__.__name__
        for thread in thread_pool:
            thread.join()

        raise exc

if __name__ == "__main__":
    main()

---

Why is the KeyboardInterrupt not catched? What is the proper way of catching exceptions in a threaded program and tear down the complete process?

Answer 1

KeyboardInterrupt is a special exception; like MemoryError, GeneratorExit and SystemExit, it does not derive from the base Exception class.

Catching just Exception is thus not enough; you'd normally catch it explicitly:

except (Exception, KeyboardInterrupt) as exc:

However, you are also trying to catch exceptions in threads; threads have their own separate stack; you cannot just go and catch exceptions thrown in those stack in your main thread. You'd have to catch exceptions in that thread:

def runserver():
    print "Running server ..."

    global running
    running = True

    try:    
        while running:
            time.sleep(0.07)
    except (Exception, KeyboardInterrupt) as exc:
        print "Error in the runserver thread"

To handle this in a generic way and 'pass' exceptions to the main thread, you'd need some sort of inter-thread communication. See Catch a thread's exception in the caller thread in Python for a full solution to that.