Individual timeouts for concurrent.futures

Question

I see two ways to specify timeouts in concurrent.futures.

as_completed()
wait()

Both methods handle N running futures.

I would like to specify an individual timeout for each future.

Use Case:

Future for getting data from DB has a timeout of 0.5 secs.
Future for getting data from a HTTP server has a timeout of 1.2 secs.

How do I handle this with concurrent.futures? Or is this library not the right tool?

Conclusion

AFAIK the solution by mdurant is a good work-around.
I think I will use a different library the next time. Maybe asyncio has better support for this. See: https://docs.python.org/3/library/asyncio-task.html#asyncio.sleep

score 4 · Accepted Answer · answered Jul 25 '16 at 20:10

4

How about implementing your own:

wait(dbfutures + httpfutures, timeout=0.5)
[fut.cancel() for fut in bdfutures if not fut.done()]
wait(httpfutures, timeout=0.7)
[fut.cancel() for fut in httpfutures if not fut.done()]

(or a while loop with sleep/check or wait with short timeout)

answered Jul 25 '16 at 20:10

mdurant

27,272
5
45
74

Yes, I think this is the only solution. Not nice, but a working work-around. I think I will use async io the next time: https://docs.python.org/3/library/asyncio-task.html#asyncio.sleep – guettli Jul 29 '16 at 07:16

Individual timeouts for concurrent.futures

Conclusion

1 Answers1

Linked