How to catch an exception which occured in a thread?

Question

Note: there is one exact answer to my question in the duplicate (also see my answer below for the modified code). Thank you @quamrana for the pointer

Context: I have a list of methods in a class which are all started in threads. Some of these methods are expected to raise exceptions and these exceptions must be handled in the main program (= not in the method itself).

The problem: the exception is not caught and the interpretation (success / failure) is wrong as all threads are "successful".

What I thought would have worked: a try/except in which the thread is actually start().

Please note in the Traceback that both answers are (...) was successful as if the try only handled the mere fact of starting the thread (.start()) and not what is happening in the thread itself.

import threading

class Checks:

    @staticmethod
    def isok():
        print("OK")

    @staticmethod
    def isko():
        raise Exception("KO")

# db will keep a map of method names in Check with the actual (executable) method
db = {}

# getting all the methods from Checks, without the built_in ones
for check in [k for k in dir(Checks) if not k.startswith('_')]:
    # create a thread for the method
    db[check] = threading.Thread(target=getattr(Checks, check))
    try:
        # start the thread
        db[check].start()
    except Exception:
        print(f"{check} raised an exception")
    else:
        print(f"{check} was successful")

# wait for all the threads to be finished
for thread in db.keys():
    db[thread].join()

# all the threads are finished at that point
print("all threads are done")

The output:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\yop\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "C:\Users\yop\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:/Users/yop/.PyCharm2019.2/config/scratches/scratch_6.py", line 11, in isko
    raise Exception("KO")
Exception: KO

isko was successful
OK
isok was successful
all threads are done

(the Traceback will be mixed with the actual output of the program, due to the threads but the sequence is always the same)

EDIT: following up on a comment I want to highlight again that the exceptions will occur in the methods but must be caught in the main program (= not handled in the methods themselves).

In a non-threaded approach it is easy: the try/exception clause in code similar to the one above would catch them as they bubble up.

Or you can use some thread safe data structure to store errors and then read in main. — alter123, Nov 08 '19 at 09:12
@JayVasant: due to the nature of the methods, as I mentioned in the question, the exceptions must be caught in the main program (the fact that they happen trigger further actions in the real case) — WoJ, Nov 08 '19 at 09:30
@SaisivaA: I do not understand how exactly to do that? (you could maybe put that in an answer?) — WoJ, Nov 08 '19 at 09:33
Instead of placing the print function, import the module traceback. And use function traceback.format_exc(). — Saisiva A, Nov 08 '19 at 09:37
@quamrana: thank you very much. My question is indeed a duplicate (and I will mark it as it) and the answer https://stackoverflow.com/a/53779560/903011 was exactly what I was looking for. Thanks again. — WoJ, Nov 08 '19 at 12:15
Starting the thread isn't what fails, so a try-except that only catches exceptions from `start` is never going to work. — user2357112, Nov 08 '19 at 12:15
Can you point out what could go wrong in case of this ans: https://stackoverflow.com/a/58764003/7870866 — alter123, Nov 08 '19 at 12:46
@JayVasant: I specifically mentioned (twice: once in the initial question, and then another time in the extra edit) in my question that the exceptions must be handled at the level of the main program, not in the methods. Your answer does the `try` in the method. — WoJ, Nov 08 '19 at 12:49

score 0 · Answer 1 · answered Nov 08 '19 at 09:41

create a queue to catch exceptions.

errors = queue.Queue()

def threaded_func():
  try:
    # perform some task
  except:
    errors.put(  
      # push into queue as required
    )

def main():
  while True and threads_running:
   if errors.__len__():
     error_in_main = errors.pop()
     # handle the error as required.

Using this manner you can almost immideately catch errors in main thread and perform operations as required.

WoJ · Accepted Answer · 2019-11-08T12:36:23.620

The exact answer to my problem was given in an answer to another question (I marked mine as duplicate, but it is really that particular answer which is perfect in my context).

The code in my question modified to account for the solution:

import concurrent.futures

class Checks:

    @staticmethod
    def isok():
        print("OK")

    @staticmethod
    def isko():
        raise Exception("KO")

# db will keep a map of method namles in Check with the actual (executable) method
db = {}

with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
    for check in [k for k in dir(Checks) if not k.startswith('_')]:
        db[check] = executor.submit(getattr(Checks, check))
    for future in concurrent.futures.as_completed([db[k] for k in db.keys()]):
        if future.exception():
            print(f"{str(future.exception())} was raised (I do not know where, somewhere)")
        else:
            print("success (again, I do not know where, exactly)")

# all the threads are finished at that point
print("all threads are done")

I do not know how to get the name of the future which raised the exception (I will ask a new question about that)

How to catch an exception which occured in a thread?

2 Answers2

Linked