Break function and move on after certain time has elapsed - Python

Question

I have been trying to create an timeout that tells a section of my code to move on if it hasn't reached a result within a certain amount of time.

I tested the code from the main answer posted here: break the function after certain time , and it works perfectly however when I implemented it with my code it does not work.

The TimeoutException is never being triggered no matter how long it takes for the block of code within the 'Try' to execute. The timer within this block of code consistently prints out as greater than 5 seconds even though the print action should not even be called if the code takes longer than the 5 seconds allowed by signal.alarm(5).

Can you think of any reasons that the TimeoutException is not being triggered and how I can fix it?

import signal
import time

def evaluateList(dataDictionaryList):
    newDataDict = []
    count = 0


    class TimeoutException(Exception):   
        pass

    def timeout_handler(signum, frame):   
        raise TimeoutException

    signal.signal(signal.SIGALRM, timeout_handler)


    for dataDictionary in dataDictionaryList:

        signal.alarm(5)

        try:
            startTime = time.time()
            newDataDict.append(functionThatMightHang(dataDictionary))
            print(count + 1, '  Time to evaluate: ', time.time() - startTime)
            count+=1

        except TimeoutException:
            print('took too long')
            continue
        else:
            signal.alarm(0)

    return newDataDict

score 2 · Accepted Answer · answered May 28 '17 at 11:31

It should work indeed - except if some code block inside functionThatMightHang is itself capturing and swallowing your TimeoutException. This kind of thing will happen if there is a try/except block capturing a bare Exception in there, and just ignoring it and continuing the calculation. (That is a poor programming practice, and even listed as so in the zen of Python).

If you have access to that function source code, hunt for try/excecpt blocks and include prints to check it, and capture specific exceptions instead of bare except.

If you can't dig into that code - you can make a single shot attempt to change the base of your TimeoutException to BaseException (in place of Exception) .

In any case, your does not seem to be the best approach to the problem you have at hand - signals really don't go along well with a lot of type of codes, including multi-threaded or other event-loop based code.

Moreover, even if your handler properly raises a Python exception at the right time, if your long-running function is native code, Python will not itself handle the exception (and thus switch execution to the except block) until the native-code function returns. I ruled that out as your (sole) problem because in that case the following print-statement would not run - it would take more than 5 seconds, and them execute the except block.

Real fix

You might want to try to run your long-term function by using Python's concurrent.future module. In that way you can run your long-term function in another thread, in a more or less transparent way (or even in another process, which will take advantage of multiple cores), and specify a timeout parameter for the execution - Python's concurrent.future machinery will deal with stopping the calculation by itself.

This example works on the interactive interpreter, and raises TimeoutError (the one used by concurrent.futures, not a custom one):

from concurrent.futures import ThreadPoolExecutor
import time

def long():
    time.sleep(3)

p = ThreadPoolExecutor(1)

f = p.submit(long); f.result(timeout=1)

Break function and move on after certain time has elapsed - Python

1 Answers1

Real fix