Python use signal alarm in order to limit function execution time dosnt always stop the method

Question

I need to limit function execution time, so i followed Josh Lee answer

try:
    with time_limit(10):
        long_function_call()
except TimeoutException, msg:
    print "Timed out!"

where long_function_call() is a selenium webdriver function that interact with a page and do some operations.

def long_function_call(self, userName, password):
    driver = self.initDriver()
    try: 
         driver.get("https://yyyy.com")
         time.sleep(2)
         if not self.isHttps(driver.current_url):
              isHttps = False             
         driver.find_element_by_id("i015516").clear()
         time.sleep(5)
         if 'https://yyy.com' not in driver.current_url:
             self.raiseFailedToLogin('yyy')                    
    except Exception as e:
        self.raiseException('yyy',e)
    finally:
        driver.close()
        driver.quit()
    return 'yyyy'

In most cases , when function execution time exceed the signal timeout signal was sent and method stopped, but in some cases the method exceed the timeout and didnt stop. it seems that selenium is hang.(the firefox is open and nothing is done in it).

I tried to pause debugger in these cases , but pause didn't show me where it hang. If i close the selenium firefox than the debug pause stop on this method:

_read_status [httplib.py:366]   
begin [httplib.py:407]  
getresponse [httplib.py:1030]   
do_open [urllib2.py:1180]   
http_open [urllib2.py:1207]
def _read_status(self):
    # Initialize with Simple-Response defaults
    line = self.fp.readline()
    if self.debuglevel > 0:  ################Hang here

Any idea why in some cases signal alarm with selenium didnt work? (i dont think they catch interrupt).

it might be ["SIGALRM itself might interrupt the call that's blocking--but socket code typically simply retries after an EINTR"](http://stackoverflow.com/a/1114567/4279). Though current socketmodule.c checks for signals and [I can't reproduce it (alarm works) on my machine on Python 2.4-3.3](https://gist.github.com/4069063) — jfs, Nov 13 '12 at 23:30
@J.F.Sebastian Thanks for your comment. can you please copy it as an answer. It seems to have good information , so it should be as an answer with comment capabilities. I didnt clearly understand what to do with your comment. How to solve this problem? — Avihai Marchiano, Nov 14 '12 at 09:26
Have you tried running [the test I've linked](https://gist.github.com/4069063)? My comment would be the answer if the test did hang on your machine. — jfs, Nov 14 '12 at 12:49
@J.F.Sebastian i did ran , i am getting py27: commands succeeded. As i told the alarm work in most of the cases , but in some case when selenium do somethings it didnt work — Avihai Marchiano, Nov 14 '12 at 14:19

Marwan Alsabbagh · Answer 1 · 2012-11-13T21:01:45.760

This is a very interesting problem you are facing. I've created an example which can demonstrate some of the issues you may face when using with time_limit. If you run the code below you may expect that after 1 second a TimeoutException will be raised which will then exit python and the running thread as well as the xterm should all exit. What happens is after one second the TimeoutException is raised and you will see the "Timed out!" appear in the terminal but both the thread and xterm will continue their execution. This maybe the type of scenario you are facing with selenium. I don't know how selenium is implemented but the firefox process must be spawned in a way similar to how xterm is in the example.

simple approach

import time, os, sys, subprocess, threading

def worker():
    for i in range(30):
        print i
        time.sleep(0.1)

try:
    with time_limit(1):
        threading.Thread(target=worker).start()
        subprocess.check_output('/usr/bin/xterm')
except TimeoutException, msg:
    print "Timed out!"

a potential solution to this problem is the following code. It will kill all children of this python program which would kill xterm in this case, and then it would kill itself. This is of course a forceful way to exit but would guarantee everything ended on timeout.

kill them all

subprocess.call('pkill -P %s' % os.getpid(), shell=True)
subprocess.call('kill -9 %s' % os.getpid(), shell=True)

taking into account your comments below another approach would be to have another thread that would perform the kill operations if an operation in the main thread exceeded the specified timeout. This is an implementation of that approach with an example call.

thread that watches and kills on timeout

import time, os
from subprocess import check_output, call
from threading import Thread
from contextlib import contextmanager

def wait_thread(active, duration):
    start = time.time()
    while active[0] and time.time() - start < duration:
        time.sleep(0.1)
    if active[0]:
        call('pkill -P %s' % os.getpid(), shell=True)
        call('kill -9 %s' % os.getpid(), shell=True)

@contextmanager
def wait(duration):
    active = [True]
    Thread(target=wait_thread, args=(active, duration)).start()
    yield
    active[0] = False

with wait(1):
    time.sleep(0.5)
    print 'finished safely before timeout'
with wait(1):
    call('/usr/bin/xterm')

In my case i dont see the timeout raise from the long method. In some scenarios i dont see the signal fired. I dont catch timeout for these cases. I tried to consider alternative way to set the timeout like the thread solution in the link i posted , but i saw critisim on using thread solution for this and i didnt find another solution for this — Avihai Marchiano, Nov 13 '12 at 19:28

Python use signal alarm in order to limit function execution time dosnt always stop the method

1 Answers1