python threading: how to interrupt the main thread and make it do something else

Question

Using python threading, is it possible to interrupt the main thread without terminate it and ask it to do something else?

For example,

def watch():
    while True:
        sleep(10)
        somethingWrong = check()
        if somethingWrong:
            raise Exception("something wrong")
try:
    watcher = threading.Thread(target=watch)
    watcher.start()
    doSomething() # This function could run several days and it cannot detect something wrong within itself, so I need the other thread watcher to check if this function perform well. In case of malfunction, the watcher should interrupt this function and ask the main thread to doSomethingElse
except:
    doSomethingElse()

In this program the main thread is not affected when an exception is raised in the thread watcher and keep doSomething. I want the exception raised in the child thread to propogate to the main thread and make the main thread doSomethingElse.How can I do this?

Please note that many questions similar to this and asked on this platform are in fact irrelevant. In those cases, the main thread is waiting for the message from the child thread. But in my case, the main thread is doing something. The reason for which those anwsers are not applicable here is that the main thread cannot do something and listen to the child thread at the same time.

Does this answer your question? [Catch a thread's exception in the caller thread?](https://stackoverflow.com/questions/2829329/catch-a-threads-exception-in-the-caller-thread) — VPfB, Jul 20 '22 at 11:51
That answer seems not relevent to my question, because it requires the main thread to wait until the child thread rasies an exception. In my case, the main thread is doing something and is not waiting. — Yufei, Jul 21 '22 at 02:06
Do you want the main thread to be interrupted in whatever it is doing (writing a file, updating a user interface, etc.) by the exception from the thread? With no way to resume what it was doing? — Grismar, Jul 21 '22 at 03:24
The way to resume what it was doing is in fact in `doSomethingElse`. The real situation is that I want to download many files from a ftp server. The download, which is `doSomething` in the above code, is fast at first but slows down afterwards even to several Bytes/s. In this case, I want a watcher to check every 10s if the download slows down and in such case to interrupt the ftp connection, reconnect to the ftp server and resume the download, which is `doSomethingElse` in the above code. — Yufei, Jul 21 '22 at 04:44

Veysel Olgun · Accepted Answer · 2022-07-21T14:50:59.190

I think below code will solve your problem.

We need to konow that :

Python signal handlers are always executed in the main Python thread of the main interpreter, even if the signal was received in another thread

We send our process SIGUSR1 signal in the subthread.

The SIGUSR1 and SIGUSR2 signals are set aside for you to use any way you want. They’re useful for simple interprocess communication, if you write a signal handler for them in the program that receives the signal.

In our case this signals will be handled in the main thread. When subthread send signal to own process, signal handler run in the main thread and we will throw an exception in the signal handler. After that exception, your do_Something() catch this exception and will enter to SubThreadException block, in this block you can do some cleanup. After the cleanup() we throw the SubThreadException again because we need to run your doSomethingElse() func.

class SubThreadException(Exception):
    pass

def usr1_handler(signum, frame):
    raise SubThreadException()

signal.signal(signal.SIGUSR1,usr1_handler) 

def watch():
    while True:
        sleep(10)
        somethingWrong = check()
        if somethingWrong:
            os.kill(os.getpid(),signal.SIGUSR1)
                    
def doSomething():
    try:
        #your work here
    except SubThreadException:
        cleanup() #your cleanup, like close the file,flush the buffer
        raise SubThreadException
        
try:
    watcher = threading.Thread(target=watch)
    watcher.start()
    doSomething() # This function could run several days and it cannot detect something wrong within itself, so I need the other thread watcher to check if this function perform well. In case of malfunction, the watcher should interrupt this function and ask the main thread to doSomethingElse
except SubThreadException:
    doSomethingElse()

But there are some cases as noted in the python signal module doc :

If a signal handler raises an exception, the exception will be propagated to the main thread and may be raised after any bytecode instruction. Most notably, a KeyboardInterrupt may appear at any point during execution. Most Python code, including the standard library, cannot be made robust against this, and so a KeyboardInterrupt (or any other exception resulting from a signal handler) may on rare occasions put the program in an unexpected state

Your solution works well for most cases, but SubThreadException doesn't get raised if my code is waiting on input() or time.sleep(X). The exception is only raised after these functions complete execution. Any way to solve this? By contrast, KeyboardInterrupt that is raised after pressing Ctrl-C doesn't have these problems. — MMM, Dec 22 '22 at 18:49
@MMM you mean when `time.sleep()` or `input()` in`doSomething()` func ? — Veysel Olgun, Dec 24 '22 at 19:27
I think you are wrong here because, when main thread got `SIGUSR1` signal, it will be interrupted whatever it doing(If we don't ignore this signal). After interrupted, it will jump to `usr1_handler()` function(triggered by OS kernel) I tested it with `time.sleep()` and `input()`. — Veysel Olgun, Dec 26 '22 at 13:08

python threading: how to interrupt the main thread and make it do something else

1 Answers1

Linked