Compare methods to terminate running a function after time period

Question

I have a program, which opens a lot of urls and downloads pictures . I have a function of the program, which manages link's opening and pictures downloading, which contains a for loop and performs some operations on the priority queue. I want to run this function, but no longer than the set time period. For example if this function is running longer than 1 hour I want to terminate it and run the rest of the program (other functions). I was trying to find some solutions, and I found two question here on stack. The first solution use only time module First solution

The second use also the multiprocessing module Second solution. Can some one suggest which one will be more appropriate to use in my program? I will write a pseudocode of my function:

def fun():
 for link in linkList:
   if link not in queue:
     queue.push(link)
   else:
     queue.updatePriority(link)
 if queue:
   top = queue.pop()
   fun(top)

This function is called in other function: def run(startLink): fun(startLink)

And the run() function is called in other module. Which method is better to use with a program which contains a lot of modules and performs a lot of

score 1 · Answer 1 · answered Aug 22 '14 at 12:29

1

The asyncio module is ideal for this task.

You can create a future, then use asyncio.wait which supports a timeout parameter.

answered Aug 22 '14 at 12:29

simonzack

19,729
13
73
118

@But I want use the built-in modules – Ziva Aug 22 '14 at 12:31
The asyncio module is built-in. – simonzack Aug 22 '14 at 12:31
2

The OP is using 2.7 - see the tags. – Maciej Gol Aug 22 '14 at 12:41

score 1 · Answer 2 · answered Aug 22 '14 at 18:51

Using multiprocessing here would be a little bit tricky, because fun is consuming a priority queue (I'm assuming a Queue.PriorityQueue) that is coming from some other part of the program. That queue cannot easily be passed between processes - you would need to create a custom multiprocessing.BaseManager subclass, register the Queue.PriorityQueue class with it, and start up the Manager server, instantiate a PriorityQueue on the server, and use a Proxy to that instance everywhere you interact with the queue. That's a lot of overhead, and also hurts performance a bit.

Since it appears you don't actually want any concurrency here - you want the rest of the program to stop while fun is running - I don't think there's a compelling reason to use multiprocessing. Instead, I think using the time-based solution makes more sense.

Compare methods to terminate running a function after time period

2 Answers2