python flask: asynchronous requests

Question

In a flask view I receive data via an API call and this data has to be passed to an external API that is sometimes very slow.

So I want the view to return a positive status code while the request is being handled async.

I have tried with requests-futures and have difficulties with the callback:

def bg_cb(sess, resp):
    print(resp.text)

@app.route('/incomingdata', methods=['POST',])
def clients():

    (... process incoming POST data and create outgoing API call. here I inserted a demo call to httpbin.org that simulates a very slow API ...)

    from requests_futures.sessions import FuturesSession
    session = FuturesSession()
    future = session.get('http://httpbin.org/delay/3', background_callback=bg_cb)
    response = future.result()

    return jsonify({'status': 'ok'}), 200

Unfortunately the above code will wait with the return until the callback has been processed. Here that is 3 seconds.

How can I achieve the result, that the view returns response with 200 immediately and after 3 seconds the callback function is called.

Thank you in advance!

`requests-futures` is not going to help you with a single API call. Also, what would you be returning other than 200 during the 3 second wait? — roganjosh, Nov 01 '18 at 15:27
Or does the caller only need to know that you received the request, and there is no actual data to return to them? — roganjosh, Nov 01 '18 at 15:28
exactly. The caller only needs to know the data has been delivered. always 200. — caliph, Nov 01 '18 at 15:50
the callback function will have a lot more business logic... — caliph, Nov 01 '18 at 15:50

score 0 · Answer 1 · answered Nov 06 '18 at 09:36

You shouldn't use result() if you need just to call something asynchronously. The result() method will wait until background_callback will be finished. Just an example:

def bg_cb(sess, resp):
    print('done')

# in view:
print('call async...')
session.get('http://httpbin.org/delay/3', background_callback=bg_cb)

return jsonify({'status': 'ok'}), 200

Call endpoint, you will see response without delay. Console output:

call async...
# after few seconds
done

Now let's wait until callback will be finished:

future = session.get('http://httpbin.org/delay/3', background_callback=bg_cb)
print('wait result...')
response = future.result()
print('after result...')

Call endpoint, you will see response with delay. Console output:

wait result...
done
after result...

So, you don't need result() if you need just run something asynchronously.

Also you can use rq or celery if you need specific asynchronous processing.

Hope this helps.

I tried your code but got runtime error "get() got an unexpected keyword argument 'background_callback'" — Zvi, May 13 '21 at 12:15
@Zvi try to update `package`. [check](https://github.com/ross/requests-futures) when `background_callback` was added. — Danila Ganchar, May 13 '21 at 12:22
After update got `background_callback` is deprecated and will be removed in 1.0, use `hooks` instead, but the program did not crash this time. Switched to 'hooks' and it worked with no errors or warnings — Zvi, May 14 '21 at 13:29

python flask: asynchronous requests

1 Answers1