1

I have a python program which operates an external program and starts a timeout thread. Timeout thread should countdown for 10 minutes and if the script, which operates the external program isn't finished in that time, it should kill the external program.

My thread seems to work fine on the first glance, my main script and the thread run simultaneously with no issues. But if a pop up window appears in the external program, it stops my scripts, so that even the countdown thread stops counting, therefore totally failing it's job.

I assume the issue is that the script calls a blocking function in API for the external program, which is blocked by the pop up window. I understand why it blocks my main program, but don't understand why it blocks my countdown thread. So, one possible solution might be to run a separate script for the countdown, but I would like to keep it as clean as possible and it seems really messy to start a script for this.

I have searched everywhere for a clue, but I didn't find much. There was a reference to the gevent library here: background function in Python , but it seems like such a basic task, that I don't want to include external library for this.

I also found a solution which uses a windows multimedia timer here, but I've never worked with this before and am afraid the code won't be flexible with this. Script is Windows-only, but it should work on all Windows from XP on.

For Unix I found signal.alarm which seems to do exactly what I want, but it's not available for Windows. Any alternatives for this?

Any ideas on how to work with this in the most simplified manner?

This is the simplified thread I'm creating (run in IDLE to reproduce the issue):

import threading
import time

class timeToKill():
    def __init__(self, minutesBeforeTimeout):
        self.stop = threading.Event()
        self.countdownFrom = minutesBeforeTimeout * 60

    def startCountdown(self):
        self.countdownThread= threading.Thread(target=self.countdown, args=(self.countdownFrom,))
        self.countdownThread.start()

    def stopCountdown(self):
        self.stop.set()
        self.countdownThread.join()

    def countdown(self,seconds):
        for second in range(seconds):
            if(self.stop.is_set()):
                break
            else:
                print (second)
                time.sleep(1)

timeout = timeToKill(1)
timeout.startCountdown()
raw_input("Blocking call, waiting for input:\n")
Community
  • 1
  • 1
Anja V.
  • 115
  • 1
  • 1
  • 12
  • Please help us reproduce the problem by giving us some (hopefully) small but complete code example showing the issue. – Alfe Nov 05 '13 at 08:50
  • 1
    How exactly do you "operate" the external program (call the blocking function)? – martineau Nov 05 '13 at 09:08
  • I added a sample blocking call to my example. The issue is now reproducible. I want my timer thread to continue counting while I wait for input. – Anja V. Nov 05 '13 at 09:50
  • As I noticed, the raw_input is only blocking in IDE and not if you run it normally. So, for the sake of the example, please run it in IDLE. – Anja V. Nov 05 '13 at 10:11
  • 1
    This isn't *quite* a bug, but in your code you're using `self.countdown` to mean two different things at different times. Before `startCountdown` runs, it's a method, afterwards it's a thread instance. You probably should pick a different name for one of those uses. It won't change things though, since you only use the method as the target for the thread, and it is still available at that point. – Blckknght Nov 05 '13 at 10:48
  • Yeah, I agree it was dirty, missed it. I renamed the thread now. Any ideas? – Anja V. Nov 05 '13 at 11:05
  • Do you use `subprocess.Popen()` to start the external program (if not, why not)? – jfs Nov 05 '13 at 14:19
  • No, I use API that was provided with that software. – Anja V. Nov 05 '13 at 14:38
  • How do you run the code? It should work as is (if it doesn't work in IDLE and you need to run it in IDLE then specify it). – jfs Nov 05 '13 at 15:33
  • I run it in IDLE (the above sample). I added a note next to the sample for clarity. – Anja V. Nov 05 '13 at 18:47

2 Answers2

1

For quick and dirty threading, I usually resort to subprocess commands. it is quite robust and os independent. It does not give as fine grained control as the thread and queue modules but for external calls to programs generally does nicely. Note the shell=True must be used with caution. 

#this can be any command
p1 = subprocess.Popen(["python", "SUBSCRIPTS/TEST.py", "0"], shell=True)

#the thread p1 will run in the background - asynchronously.  If you want to kill it after some time, then you need 

#here do some other tasks/computations
time.sleep(10)

currentStatus = p1.poll()
if currentStatus is None: #then it is still running
  try:
    p1.kill() #maybe try os.kill(p1.pid,2) if p1.kill does not work
  except:
    #do something else if process is done running - maybe do nothing?
    pass
Paul
  • 7,155
  • 8
  • 41
  • 40
  • Currently I'm using a subprocess similarly to your example, although I hoped I could avoid it. May I ask why is shell=True necessary in your sample? – Anja V. Nov 05 '13 at 14:35
  • @Anja, The shell=True is not necessary. I often include from rote. Other than subprocess you can always use the threads or queue modules – Paul Nov 05 '13 at 15:20
  • 2
    it is incorrect to use `shell=True` with a list argument unless you want to pass additional arguments to the shell (it just fails on POSIX systems where it is equivalent to [`Popen(['/bin/sh', '-c'] + list_arg)`](http://hg.python.org/cpython/file/6ff5bd97635f/Lib/subprocess.py#l1192)). – jfs Nov 05 '13 at 15:20
  • Yes, I was trying to use threads, but my timeout thread got blocked, stopped counting. Do you maybe understand why the timeout thread in my example stops when the main program reaches a blocking call? – Anja V. Nov 05 '13 at 15:25
1

One possible explanation for a function call to block another Python thread is that CPython uses global interpreter lock (GIL) and the blocking API call doesn't release it (NOTE: CPython releases GIL on blocking I/O calls therefore your raw_input() example should work as is).

If you can't make the buggy API call to release GIL then you could use a process instead of a thread e.g., multiprocessing.Process instead of threading.Thread (the API is the same). Different processes are not limited by GIL.

jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • I was wondering why my attempt was failing and this explains the issue. So far I have the solution with subprocess module, but multiprocessing seems better when dealing with several processes. Thanks for the idea! – Anja V. Nov 05 '13 at 18:44
  • @AnjaV.: if it is indeed the issue then the correct solution is to fix the upstream API for the external program to release GIL when it invokes blocking system calls. `multiprocessing.Process` is just a workaround. – jfs Nov 05 '13 at 19:58