Running background Celery task in Flask

Question

Problem has been updated to include progress made

I have the following code and my celery tasks kick off fine, I just don't know where I should store the async result so that I can look at it again later

#!/usr/bin/env python

"""Page views."""

from flask import render_template, request
from flask import Flask

from celerytest import add

from time import sleep


app = Flask(__name__)

async_res = []

@app.route('/', methods=['GET', 'POST'])
def run():
    if request.method == 'GET':
        return render_template("template.html")
    else:
        form = request.form
        n1 = str(form.get("n1"))
        n2 = str(form.get("n2"))
        aysnc_res = add.delay(n1,n2)
        return render_template("loading.html")

@app.route('/loading')
def check_if_complete():
    if async_res.ready() == True:
        return render_template("template2.html", val=async_res.get())
    else:
        sleep(5)
        return render_template("loading.html")


if __name__ == '__main__':
    app.run()

It appears that storing async_res as a global variable in my flask app causes server errors. So what's the best method of storing the result so that I can check on it in my "loading page"

Typically, you wouldn't start a Celery worker from inside Flask. You would start the Celery worker separately from starting Flask (see the [Worker Guide](http://docs.celeryproject.org/en/latest/userguide/workers.html)). When the task is run, you would start a task as shown in the [Task Guide](http://docs.celeryproject.org/en/latest/userguide/tasks.html). — Mark Hildreth, Aug 08 '13 at 19:44
@MarkHildreth that only partially answers my question of how I kick off the Celery task, how do I know when it's complete and force the page to render the results? — sedavidw, Aug 08 '13 at 20:10
You kick off a task as shown in the Task Guide that I linked to. You know it's complete because your task makes some change as it's last step (such as changing the status flag of a record in a database). You can check that status flag in web requests. How to force the page to reload is completely up to you; there are a many different ways (some of which I listed in the [previous answer](http://stackoverflow.com/questions/18127128/time-out-issues-with-chrome-and-flask)). I would recommend playing with Celery by itself until you understand what it can do. — Mark Hildreth, Aug 08 '13 at 21:29
@MarkHildreth I've made some progress in learning celery, I can kick off workers and start tasks no problem. And using them in interactive mode is intuitive. I'm having trouble finding where I should store my async result in my Flask app so that I can check on it later. I've updated the question to reflect the new status — sedavidw, Aug 09 '13 at 03:15
What is the error message you're getting? Another option is, in your long-running task, save the result of the task (or the status of the task) to a database and read the results from the database. — Mark Hildreth, Aug 09 '13 at 12:16
I actually ended up saving the ID in a session, seemed like the more efficient solution. Thanks for all your help! — sedavidw, Aug 09 '13 at 13:26
Sounds good. Make sure you create an answer for this question with what you did. — Mark Hildreth, Aug 09 '13 at 13:51
@MarkHildreth the last (I think) problem I'm having is an import error in Celery (http://stackoverflow.com/questions/18148210/trouble-importing-into-my-celery-task-file). You seem pretty knowledgeable on the subject so if you have a few minutes to look at that question I'd appreciate it — sedavidw, Aug 09 '13 at 14:34

score 3 · Accepted Answer · answered Aug 09 '13 at 13:55

I ended up being able to save the ID of my task in the session dictionary in Flask

See code below:

#!/usr/bin/env python

"""Page views."""

from flask import render_template, request
from flask import Flask

from celerytest import add

from time import sleep


app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def run():
    if request.method == 'GET':
        return render_template("template.html")
    else:
        form = request.form
        n1 = str(form.get("n1"))
        n2 = str(form.get("n2"))
        aysnc_res = add.delay(n1,n2)
        session['TASK_ID'] = async_res.id
        return render_template("loading.html")

@app.route('/loading')
def check_if_complete():
    aysnc_res = session['TASK_ID']
    if async_res.ready() == True:
        return render_template("template2.html", val=async_res.get())
    else:
        sleep(5)
        return render_template("loading.html")


if __name__ == '__main__':
    app.run()

You probably don't want to "sleep" in `check_if_complete()`. — Mark Hildreth, Aug 09 '13 at 15:20

Running background Celery task in Flask

1 Answers1