How to get a return value from a thread, asynchonously?

Question

My problem: starting a threaded function and, asynchronously, act upon the returned value

I know how to:

start a threaded function with threading. The problem: no simple way to get the result back
get the return value from a threaded function. The problem: it is synchronous

What I would like to achieve is similar to JavaScript's

aFunctionThatReturnsAPromise()
  .then(r => {// do something with the returned value when it is available})
// the code here runs synchronously right after aFunctionThatReturnsAPromise is started

In pseudo-Python, I would think about something like (modifying the example from the answer to the linked thread)

import time
import concurrent.futures

def foo(bar):
    print('hello {}'.format(bar))
    time.sleep(10)
    return 'foo'

def the_callback(something):
    print(f"the thread returned {something}")

with concurrent.futures.ThreadPoolExecutor() as executor:
    # submit the threaded call ...
    future = executor.submit(foo, 'world!')
# ... and set a callback
future.callback(the_callback, future.result())  # ← this is the made up part
# or, all in one: future = executor.submit(foo, 'world!', callback=the_callback) # in which case the parameters probably would need to be passed the JS way
# the threaded call runs at its pace
# the following line is ran right after the call above
print("after submit")
# after some time (~10 seconds) the callback is finished (and has printed out what was passed to it)
# there should probably be some kind of join() so that the scripts waits until the thread is done

I want to stay if possible with threads (which do things at their own pace and I do not care when they are done), rather than asyncio (where I have to explicitly await things in a single thread)

martineau · Answer 1 · 2021-11-29T09:19:29.217

You can use concurrent.futures.add_done_callback as shown below. The callback must be a callable taking a single argument, the Future instance — and it must get the result from that as shown. The example also adds some additional information to it which the callback function uses for printing its messages.

Note that the callback function(s) will be called concurrently, so the usual mutex precautions should be taken if there are shared resources involved. This wasn't been done in the example below, so sometimes the printed output will be jumbled.

from concurrent import futures
import random
import time

def foo(bar, delay):
    print(f'hello {bar} - {delay}')
    time.sleep(delay)
    return bar

def the_callback(fn):
    if fn.cancelled():
        print(f'args {fn.args}: canceled')
    elif fn.done():
        error = fn.exception()
        if error:
            print(f'args {fn.args}: caused error {error}')
        else:
            print(f'args {fn.args}: returned: {fn.result()}')

with futures.ThreadPoolExecutor(max_workers=2) as executor:
    for name in ('foo', 'bar', 'bas'):
        delay = random.randint(1, 5)
        f = executor.submit(foo, name, delay)
        f.args = name, delay
        f.add_done_callback(the_callback)

print('fini')

Sample output:

hello foo - 5
hello bar - 3
args ('bar', 3): returned: bar
hello bas - 4
args ('foo', 5): returned: foo
args ('bas', 4): returned: bas
fini

score 0 · Answer 2 · answered Apr 09 '21 at 08:39

You can use add_done_callback of concurrent.futures library, so you can modify your example like this:

def the_callback(something):
    print(f"the thread returned {something.result()}")
 
with concurrent.futures.ThreadPoolExecutor() as executor:
    future = executor.submit(foo, 'world!')
    future.add_done_callback(the_callback)

How to get a return value from a thread, asynchonously?

2 Answers2