Cleanly stopping a multiprocessing.Process - KeyboardInterrupt escapes in Windows

Question

I'm using multiprocessing to spawn a task (multiprocessing.Process) that can be stopped (without cooperation from the task itself, e.g.: without using something like multiprocessing.Event to signal the task to gracefully stop).

Since .terminate() (or .kill()) won't stop it cleanly (the finally: clause won't execute), I thought I would use os.kill() to emulate a CTRL+C event:

from multiprocessing import Process
from time import sleep
import os
import signal

def task(n):
    try:
        for i in range(n):
            sleep(1)
            print(f'task: i={i}')
    finally:
        print('task: finally clause executed!')
        return i

if __name__ == '__main__':
    t = Process(target=task, args=(10,))
    print(f'main: starting task...')
    t.start()
    sleep(5)
    for i in ('CTRL_C_EVENT', 'SIGINT'):
        if hasattr(signal, i):
            sig = getattr(signal, i)
            break
    print(f'main: attempt to stop task...')
    os.kill(t.pid, sig)

The finally: clause executes on both Windows, macOS and Linux; hoever on Windows it additionally spits out the error:

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "c:\Python38\lib\multiprocessing\util.py", line 357, in
_exit_function
    p.join()
  File "c:\Python38\lib\multiprocessing\process.py", line 149, in join
    res = self._popen.wait(timeout)
  File "c:\Python38\lib\multiprocessing\popen_spawn_win32.py", line 108, in wait
    res = _winapi.WaitForSingleObject(int(self._handle), msecs)
KeyboardInterrupt

while on macOS and Linux it only print the messages meant to be printed.

What happens if you make the process `daemon=True`? Note that you then need to keep the main process alive as long as the daemon processes are supposed to run. — zardosht, Aug 06 '21 at 11:18
When setting `daemon=True` the same error is printed, and additionally, the `finally:` clause is not executed. — fferri, Aug 06 '21 at 11:48

score 3 · Answer 1 · answered Aug 09 '21 at 14:02

It seems CTRL_C_EVENT in Windows is also propagated from the child process to the parent process. See for example this related question.

I added some book keeping code and a try...except block to the code. It shows what happens, and that the KeyboardInterrupt needs to be caught on parent process as well.

from multiprocessing import Process
from time import sleep
import os
import signal

def task(n):
    try:
        for i in range(n):
            sleep(1)
            print(f'task: i={i}')
    except KeyboardInterrupt:
        print("task: caught KeyboardInterrupt")
    finally:
        print('task: finally clause executed!')
        return i

if __name__ == '__main__':
    try:
        t = Process(target=task, args=(10,))
        print(f'main: starting task...')
        t.start()
        sleep(5)
        for i in ('CTRL_C_EVENT', 'SIGINT'):
            if hasattr(signal, i):
                sig = getattr(signal, i)
                break
        print(f'main: attempt to stop task...')
        os.kill(t.pid, sig)

    finally:
        try:
            print("main: finally in main process. Waiting for 3 seconds")
            sleep(3)
        except KeyboardInterrupt:
            print("main: caught KeyboardInterrupt in finally block")

It prevents the error and produces the following output:

main: starting task...
task: i=0
task: i=1
task: i=2
task: i=3
main: attempt to stop task...
main: finally in main process. Waiting for 3 seconds
task: caught KeyboardInterrupt
main: caught KeyboardInterrupt in finally block
task: finally clause executed!

Cleanly stopping a multiprocessing.Process - KeyboardInterrupt escapes in Windows

1 Answers1