Aborting a loop after some time passes

Question

I have a code which may run into long loops. These are not infinite loops but certain inputs may cause the loop to continue for extended periods. I wish to use an interrupt in case a loop goes too long. With this interrupt, the loop will re-evaluate its inputs, so a keyboard interrupt is not what I am looking for. I am using python 2.7 on Windows.

One possible approach would be polling using time.time(), yet it would be extremely inefficient. The loop itself is not long, however even in normal runs it is iterated 100k times, so I can't poll the time after each iteration, it would decrease efficiency a lot. Alternatively, I could introduce on more variable,

i=0
while i<some_very_large_number:
    function(foo, foo1, foo2)
    i=i+1

but this again, would be a mediocre solution.

In short, what I am looking for is some python equivalent to the hardware interrupt of microprocessors. I have no experience in multi-threading, so if the answer lies in multi-threading, please elaborate a little bit.

I have checked here and here, yet I do not think they answer my question. The second link could actually help yet apparently signal package is not available in Windows.

The code is long but straightforward. It basically has such a structure.

def function(foo, foo1, foo2, N):
    for i in range(N):
        performance = performance_evaluator(foo, foo1, foo2)
        if performance_takes_too_long:
             interrupt ##this if clause is the main objective of this question##
        record performance
        new_foo, new_foo1, new_foo2 = evolve(foo, foo1, foo2)

What's the problem using `time`? is it that it would add a lot more running time to your code than you want? — Akaisteph7, Jul 16 '19 at 14:06
@blackbrandt Thank you but signal documentation says it is only available in UNIX. Isn't it so? Do I mix something up? — ck1987pd, Jul 16 '19 at 14:08
"One possible approach would be polling using time.time(), yet it would be extremely inefficient" What do you mean exactly? How or why would it be inefficient? It seems that the most reasonable approach here would be to save in a variable the time at the start of the function and measure on each iteration the elapsed time, then raise and exception or something if the elapsed time is too large. Would not that work for you? — jdehesa, Jul 16 '19 at 14:08
@C.Koca are you on a Windows platform? If so, that would be important and you should put that in the original post. — blackbrandt, Jul 16 '19 at 14:08
@blackbrandt I suppose it is sort of implied in "apparently signal package is not available in Windows"... — jdehesa, Jul 16 '19 at 14:10
@blackbrandt yes, I tried to imply it in the question, but I will state it clearly. — ck1987pd, Jul 16 '19 at 14:12
@D1TTO I know it would solve my problem but it is a huge overhead to check time in every iteration. I would just let a dummy variable accumulate to a very large number rather than check and store the time. — ck1987pd, Jul 16 '19 at 14:16
I can see how calling time.time() every iteration to check passed time would be inefficient but wouldn't any added in mechanism to check elapsed time be also inefficient then? Anything keeping track of the time passed would have to be checked every iteration anyways — D1TTO, Jul 16 '19 at 14:19
@D1TTO You are right, but processor has other timers, which operate in microprocessors to run the timer interrupts. That would be exceedingly faster. — ck1987pd, Jul 16 '19 at 14:21
To be honest, I still don't see how calling `time.time()` would have any significant performance impact. Running `for i in range(1000000): time.time()` (1 million iterations) in my machine takes less than 0.1 seconds. You don't get much faster than that in Python (some nanoseconds per call), whatever you do will add up to something like that or more just due to the interpreter overhead. — jdehesa, Jul 16 '19 at 15:36

abdusco · Accepted Answer · 2019-07-16T14:19:11.960

One possible approach is to modify your script so that it takes its inputs via commandline arguments, then use subprocess module to run it with a timeout:

# manager.py
import subprocess

try:
    code = subprocess.call('python work.py 5', timeout=2)
    print('Ended with code:', code)
except subprocess.TimeoutExpired:
    print('Ended with timeout')


# work.py
import sys
from time import sleep

try:
    wait = int(sys.argv[1])
except:
    wait = 10

sleep(wait)
print(f'Waited for {wait} seconds')

output:

Ended with timeout

constt · Answer 2 · 2019-07-17T14:38:49.873

You can also execute a long-running code in a separate worker process and try to terminate it when a timeout is exceeded and the worker has not been finished yet:

import time
from multiprocessing import Process


def process_data():

    while True:
        print("processing...")
        time.sleep(1)


def main():
    worker = Process(target=process_data)
    worker.start()

    timeout = 5
    time.sleep(timeout)

    if worker.is_alive():
        print("exceeded timeout", timeout, "sec")
        print("terminate worker", worker)
        worker.terminate()

    worker.join()
    print("is worker", worker, "alive:", worker.is_alive())


if __name__ == "__main__":
    main()

Here is the output:

processing...
processing...
processing...
processing...
processing...
exceeded timeout 5 sec
terminate worker <Process(Process-1, started)>
is worker <Process(Process-1, stopped[SIGTERM])> alive: False

Aborting a loop after some time passes

2 Answers2