Ctrl-C doesn't work when using threading.Timer

Question

I'm writing a multithreaded Python app on Windows.

I used to terminate the app using ctrl-c, but once I added threading.Timer instances ctrl-c stopped working (or sometimes takes a very long time).

How could this be?
What's the relation between having Timer threads and ctrl-c?

UPDATE:
I found the following in Python's thread documentation:

Threads interact strangely with interrupts: the KeyboardInterrupt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)

Answer 1

The way threading.Thread (and thus threading.Timer) works is that each thread registers itself with the threading module, and upon interpreter exit the interpreter will wait for all registered threads to exit before terminating the interpreter proper. This is done so threads actually finish execution, instead of having the interpreter brutally removed from under them. So when you hit ^C, the main thread receives the signal, decides to terminate and waits for the timers to finish.

You can set threads daemonic (with the setDaemon method) to make the threading module not wait for these threads, but if they happen to be executing Python code while the interpreter exits, you get confusing errors during exit. Even if you cancel the threading.Timer (and set it daemonic) it can still wake up while the interpreter is being destroyed -- because threading.Timer's cancel method just tells the threading.Timer not to execute anything when it wakes up, but it has to actually execute Python code to make that determination.

There is no graceful way to terminate threads (other than the current one), and no reliable way to interrupt a thread that's blocked. A more manageable approach to timers is usually an event loop, like the ones GUIs and other event-driven systems offer you. What to use depends entirely on what else your program will be doing.

Answer 2

There is a presentation by David Beazley that sheds some light on the topic. The PDF is available here. Look around pages 22--25 ("Interlude: Signals" to "Frozen Signals").

Answer 3

This is a possible workaround: using time.sleep() instead of Timer means a "graceful shutdown" mechanism can be implemented ... for Python3 where, it appears, KeyboardInterrupt is only raised in user code for the main thread. Otherwise, it appears, the exception is "ignored" as per here: in fact it results in the thread where it occurs dying immediately, but not any ancestor threads, where problematically it can't be caught.

Let's say you want Ctrl-C responsiveness to be 0.5 seconds, but you only want to repeat some actual work every 5 seconds (work is of random duration as below):

import threading, sys, time, random

blip_counter = 0
work_threads=[]
def repeat_every_5():
    global blip_counter
    print( f'counter: {blip_counter}')
    
    def real_work():
        real_work_duration_s = random.randrange(10)
        print( f'do some real work every 5 seconds, lasting {real_work_duration_s} s: starting...')
        # in a real world situation stop_event.is_set() can be tested anywhere in the code
        for interval_500ms in range( real_work_duration_s * 2 ):
            if threading.current_thread().stop_event.is_set():
                print( f'stop_event SET!')
                return
            time.sleep(0.5)
        print( f'...real work ends')
        # clean up work_threads as appropriate
        for work_thread in work_threads:
            if not work_thread.is_alive():
                print(f'work thread {work_thread} dead, removing from list' )
                work_threads.remove( work_thread )
                
    new_work_thread = threading.Thread(target=real_work)
    # stop event for graceful shutdown
    new_work_thread.stop_event = threading.Event()
    work_threads.append(new_work_thread)
    # in fact, because a graceful shutdown is now implemented, new_work_thread doesn't have to be daemon
    # new_work_thread.daemon = True
    new_work_thread.start()
    
    blip_counter += 1
    time.sleep( 5 )
    timer_thread = threading.Thread(target=repeat_every_5)
    timer_thread.daemon = True
    timer_thread.start()
repeat_every_5()

while True:
    try:
        time.sleep( 0.5 )
    except KeyboardInterrupt:
        print( f'shutting down due to Ctrl-C..., work threads left: {len(work_threads)}')
        # trigger stop event for graceful shutdown
        for work_thread in work_threads:
            if work_thread.is_alive():
                print( f'work_thread {work_thread}: setting STOP event')
                work_thread.stop_event.set()
                print( f'work_thread {work_thread}: joining to main...')
                work_thread.join()
                print( f'work_thread {work_thread}: ...joined to main')
            else:
                print( f'work_thread {work_thread} has died' )
        sys.exit(1)

This while True: mechanism looks a bit clunky. But I think, as I say, that currently (Python 3.8.x) KeyboardInterrupt can only be caught on the main thread.

PS according to my experiments, handling child processes may be easier, in the sense that Ctrl-C will, it seems, in a simple case at least, cause a KeyboardInterrupt to occur simultaneously in all running processes.

Answer 4

Wrap your main while loop in a try except:

from threading import Timer
import time

def randomfn():
    print ("Heartbeat sent!")

class RepeatingTimer(Timer):
    def run(self):
        while not self.finished.is_set():
            self.function(*self.args, **self.kwargs)
            self.finished.wait(self.interval)

t = RepeatingTimer(10.0, function=randomfn)

print ("Starting...")
t.start()

while (True):
    try:
        print ("Hello")
        time.sleep(1)
    except:
        print ("Cancelled timer...")
        t.cancel()
        print ("Cancelled loop...")
        break

print ("End")

Results:

Heartbeat sent!
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Cancelled timer...
Cancelled loop...
End