Django: can functions within views run continuously even as other requests are made?

Question

I'm trying to create a function that, when called, will extract information from an external source at irregular (and undefined) intervals. This data will then be placed in a database for later retrieval. I want this to be then running in the background even as other page requests are made. Is this possible?

It's definitely possible, but you probably want to give some more information... Why do you want to run this function from within a view for one... What's the problem you were trying to solve with this approach? — David Miller, Aug 23 '12 at 15:47
Thanks for the reply. I'm new to django, and I didn't know that you could run functions outside of views. How would you call them in that case? Edit: actually, it would be better for it to run within a view so that I could call it with an http request. — babbaggeii, Aug 23 '12 at 15:51
If it's a function you can call it from any Python code, you just import module, then call module.yourfn() *why* is it better to call from a HTTP request - what's the context? — David Miller, Aug 23 '12 at 16:25
The connection to the external data source would need to be instantiated when the page is loaded, so I guess that makes it necessary to have it within a view? — babbaggeii, Aug 23 '12 at 16:34

score 1 · Accepted Answer · answered Aug 23 '12 at 16:10

The best way to run a Django function outside the request/response cycle is to implement it as a custom management command, which you can then set to run periodically using cron.

If you're already using it, celery supports periodic tasks using celerybeat, but this requires configuring and running the celerybeat daemon, which can be a headache. Celery also supports long-running tasks (things started in a view, but completing in their own time), as described in your question title.

Thanks - I'm going to give 'celery' a go - it looks like it'll do exactly what I want. — babbaggeii, Aug 24 '12 at 08:28

score 0 · Answer 2 · edited May 23 '17 at 12:27

Since you seem to need the function to be called when a page is loaded, you can put it inside your view as

def my_view(request):
    #Call the long running function
    long_running_function()
    #Do view logic and return
    return HttpResponse(...)

To handle the long_running_function you could use celery and create a tasks.py which implements your external data source logic. Creating tasks, adding to the queue and configuring celery is summarized here

If you just need a simpler solution for trying it out, take a look at the subprocess module.

A very similar answer here Django: start a process in a background thread?

Thanks very much - 'celery' looks perfect for this. – babbaggeii Aug 24 '12 at 08:28 — babbaggeii, Aug 24 '12 at 08:28

Django: can functions within views run continuously even as other requests are made?

2 Answers2