Rationalizing and simplifying Python 3 Keyboard Interrupt behavior in a threaded program

Question

Running the following minimized and reproducible code example, python (e.g. 3.7.3, and 3.8.3) will emit a message as follows when a first Ctrl+C is pressed, rather than terminate the program.

Traceback (most recent call last):
  File "main.py", line 44, in <module>
    Main()
  File "main.py", line 41, in __init__
    self.interaction_manager.join()
  File "/home/user/anaconda3/lib/python3.7/threading.py", line 1032, in join
    self._wait_for_tstate_lock()
  File "/home/user/anaconda3/lib/python3.7/threading.py", line 1048, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt

Only on a second Ctrl+C being pressed after that, the program will terminate.

What is the rationale behind this design? What would be an elegant way for avoiding the need for more than a single Ctrl+C or underlying signal?

Here's the code:

from threading import Thread
from queue import Queue, Empty

def get_event(queue: Queue, block=True, timeout=None):
    """ just a convenience wrapper for avoiding try-except clutter in code """
    try:
        element = queue.get(block, timeout)
    except Empty:
        element = Empty

    return element


class InteractionManager(Thread):

    def __init__(self):
        super().__init__()
        self.queue = Queue()

    def run(self):
        while True:
            event = get_event(self.queue, block=True, timeout=0.1)


class Main(object):

    def __init__(self):

        # kick off the user interaction
        self.interaction_manager = InteractionManager()
        self.interaction_manager.start()

        # wait for the interaction manager object shutdown as a signal to shutdown
        self.interaction_manager.join()


if __name__ == "__main__":
    Main()

Prehistoric related question: Interruptible thread join in Python

As you suggested, it probably is to prevent you from cancelling something you might not want to by accident. — sciencepiofficial, Jul 05 '20 at 19:25
That `Main` "class" is the worst abuse of OOP I've seen recently though.. — L3viathan, Jul 05 '20 at 19:33
You never bother to shutdown the thread. After killing the main thread, Python still waits for the ``InteractionManager`` thread to stop. The *full* traceback from double [Ctrl]+[C] reveals this immediately. Use a ``daemon`` thread if you do not want it to linger. — MisterMiyagi, Jul 05 '20 at 19:43
Thank you @MisterMiyagi, I get the general idea, although it's not as immediately intuitive to me how the second traceback reveals it nor how would the join function still be alive after the second ctrl-c or what to otherwise make of the second traceback mentioning the join function. — matanster, Jul 05 '20 at 20:10

score 2 · Answer 1 · answered Jul 05 '20 at 20:31

Python waits for all non-daemon threads before exiting. The first Ctrl+C merely kills the explicit self.interaction_manager.join(), the second Ctrl+C kills the internal join() of threading. Either declare the thread as an expendable daemon thread, or signal it to shut down.

A thread can be declared as expendable by setting daemon=True, either as a keyword or attribute:

class InteractionManager(Thread):
    def __init__(self):
        super().__init__(daemon=True)
        self.queue = Queue()

    def run(self):
        while True:
            event = get_event(self.queue, block=True, timeout=0.1)

A daemon thread is killed abruptly, and may fail to cleanly release resources if it holds any.

Graceful shutdown can be coordinated using a shared flag, such as threading.Event or a boolean value:

shutdown = False

class InteractionManager(Thread):
    def __init__(self):
        super().__init__()
        self.queue = Queue()

    def run(self):
        while not shutdown:
            event = get_event(self.queue, block=True, timeout=0.1)

def main()
    self.interaction_manager = InteractionManager()
    self.interaction_manager.start()
    try:
        self.interaction_manager.join()
    finally:
        global shutdown
        shutdown = True

Rationalizing and simplifying Python 3 Keyboard Interrupt behavior in a threaded program

1 Answers1