Flask throwing 'working outside of request context' when starting sub thread

Question

I am trying to start a new thread in Python inside of a Flask application. I am doing background work that gets triggered by the request, but I don't need to wait for the work to be done to respond to the request.

Is it possible to set the flask request in this sub-threat to the request that came in? Reason being, our ACL on our queries to our DB (mongoengine in front of mongoDB) relies on the request's user (it grabs it from flask's request object) to see if they have access to the objects, and its blowing up because the request is not available in the sub-thread.

Any thoughts would be much appreciated.

Here's pseudo code of how I am handling it now, but it is not working.

@app.route('/my_endpoint', methods=['POST'])
def my_endpoint_handler():
    #do tracking in sub-thread so we don't hold up the page
    def handle_sub_view(req):
        from flask import request
        request = req
        # Do Expensive work
    thread.start_new_thread(handle_sub_view, (request))
    return "Thanks"

If you only need the user then why not just pass the user into the sub-thread? — Mark Lavin, Mar 29 '12 at 19:08
flask provides access to the request at any time, so my Document base class has a query set manager that grabs the user from the request. The work I am doing is much high level than the query set manager, so i can not just use the user by itself — MattoTodd, Mar 29 '12 at 19:26
The request is a thread/context local so it's not available *any* time otherwise you wouldn't have this problem. I still think you should look to refactor out the dependency on the request object. — Mark Lavin, Mar 29 '12 at 19:44
by _any_ time,i just mean at any point in that thread. I would just like to set the correct request context for the sub-thread, which is not that crazy — MattoTodd, Mar 29 '12 at 19:48
@MarkLavin - yes it is typically bad policy to set the request to a thread local (say for Django), but flask does this anyways — MattoTodd, Mar 29 '12 at 19:51

score 81 · Answer 1 · answered Mar 29 '12 at 19:32

Wrap your thread code in a test_request_context so you have access to context locals:

@app.route('/my_endpoint', methods=['POST'])
def my_endpoint_handler():
    #do tracking in sub-thread so we don't hold up the page
    def handle_sub_view(req):
        with app.test_request_context():
            from flask import request
            request = req
            # Do Expensive work
    thread.start_new_thread(handle_sub_view, (request))
    return "Thanks"

Edit: it's worth pointing out that the thread will have a different context than the original request. You need to extract any interesting request data, such as the user ID, before spawning the thread. You can then grab a (different) user object in the sub-thread using the ID.

You might also want to look into abstracting this out - http://celeryproject.org/ I use it for blocking operations like sending emails. The good thing is you can use mongoDB as a backend, so you wont have to add anything to your stack. — Ross, Mar 29 '12 at 19:52

score 43 · Accepted Answer · edited Apr 09 '15 at 04:24

43

Since version 0.10 there is a supported way of doing this: http://flask.pocoo.org/docs/api/#flask.copy_current_request_context

If you want the before_request hooks to run you must call current_app.preprocess_request() inside of the decorated function.

edited Apr 09 '15 at 04:24

temoto

5,394
3
34
50

answered Jul 11 '13 at 13:25

runfalk

1,996
1
17
20

score 15 · Answer 3 · answered Nov 30 '18 at 15:01

As @runfalk pointed out, you'll need to use @copy_current_request_context. Here's a working code snippet:

import threading

from flask import request, jsonify, copy_current_request_context


@app.route('/foo')
def get_foo():
    @copy_current_request_context
    def foo_main():
        # insert your code here
        print(request.url)

    threading.Thread(target=foo_main).start()

    return jsonify({'status': 'started'})

score 5 · Answer 4 · edited May 16 '18 at 11:28

5

You can copy the desired info and pass it on:

@app.route('/my_endpoint', methods=['POST'])
def my_endpoint_handler():
    #do tracking in sub-thread so we don't hold up the page
    def handle_sub_view(data):
        # Use the data in subprocess
    data = request.get_json()  # copy the data
    thread.start_new_thread(handle_sub_view, data)
    return "Thanks"

edited May 16 '18 at 11:28

Lii

11,553
8
64
88

answered May 08 '15 at 21:45

vim

845
16
13

1

it is not handy . i must copy all data about app. – jamlee May 04 '17 at 16:07

score 2 · Answer 5 · answered Aug 18 '20 at 02:15

2

A more cleaner way is using Flask built-in executor which wraps app context, see https://flask-executor.readthedocs.io/en/latest/

answered Aug 18 '20 at 02:15

frogcdcn

391
1
4
14

That's not built-in, it's an extension? – Gasp0de Apr 19 '23 at 15:19

Flask throwing 'working outside of request context' when starting sub thread

5 Answers5

Linked