54

I have a Django website, and one page has a button (or link) that when clicked will launch a somewhat long running task. Obviously I want to launch this task as a background task and immediately return a result to the user. I want to implement this using a simple approach that will not require me to install and learn a whole new messaging architecture like Celery for example. I do not want to use Celery! I just want to use a simple approach that I can set up and get running over the next half hour or so. Isn't there a simple way to do this in Django without having to add (yet another) 3rd party package?

Marc
  • 3,386
  • 8
  • 44
  • 68
  • You could just have the server return a Request Received type of response. Then if polling or websockets aren't an option, just have the server update a percent complete value maybe every 10% or 25%. And have a designated area for the user to check any running processes, and display the percent complete. This way the browser is only hitting the server when the user goes to or refreshes the running process page. Then if a process has 100% completion have a link to the results. Just some ideas. – Ryan G Feb 21 '14 at 21:28
  • Note: The server updating the percent complete value is only on the server [eg a DB value]. This can be queried if the user goes to the given page. Also this will depend on how your infrastructure can handle concurrent connections. The running process may block, so you may need to look into something like tornado. – Ryan G Feb 21 '14 at 21:30
  • 4
    I doubt *very* much that anything you write yourself is going to be simpler than just installing Celery and having done with it. – Daniel Roseman Feb 21 '14 at 21:36
  • 3
    "I doubt very much that anything you write yourself is going to be simpler than just installing Celery and having done with it." - let's see - I can think of a lot of things simpler than Celery. It's a great bit of software but it's a large glob of code that can handle large scale, high-volume distributed tasks. It requires separate installs of a key-value store, a broker and god knows what else. If you just want to run a background task then I would look for something else. – Andy Baker Dec 12 '14 at 18:15

3 Answers3

65

Just use a thread.

import threading

t = threading.Thread(target=long_process,
                            args=args,
                            kwargs=kwargs)
t.setDaemon(True)
t.start()
return HttpResponse()

See this question for more details: Can Django do multi-thread works?

Community
  • 1
  • 1
Benjamin Toueg
  • 10,511
  • 7
  • 48
  • 79
  • 9
    I think this needs more visibility. I recently built a project around Celery and RabbitMQ when I really should have just been using threading. I built a minimal example of this here: https://github.com/nbwoodward/django-async-threading – nbwoodward Nov 15 '18 at 18:39
  • It works great, just for confirmation ...does it have any drawbacks? – itsmehemant7 Jun 04 '23 at 10:55
17

Have a look at django-background-tasks - it does exactly what you need and doesn't need any additional services to be running like RabbitMQ or Redis. It manages a task queue in the database and has a Django management command which you can run once or as a cron job.

user226114
  • 179
  • 1
  • 3
  • 4
    While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/low-quality-posts/11499101) – Rao Mar 04 '16 at 04:04
  • 3
    @Rao The link points to a module which solves the problem, not a description of how to solve the problem. I wouldn't know how else to rewrite the post, maybe you have some tips? – user226114 Mar 04 '16 at 09:31
  • 3
    This module currently only works with Django 1.4 through 1.8. – aboutaaron Aug 11 '16 at 21:46
  • 3
    We pushed a more stable and robust version (1.1.6) of django-background-tasks to the Python Package Index today. This has support for the latest greatest Django version as well. Release notes: https://github.com/arteria/django-background-tasks/releases/tag/v1.1.6 – phi Mar 29 '17 at 12:53
  • 1
    Does anybody know how to fire a background job immediately in a view, (but not wait for it to complete) using django-background-tasks? – konstunn May 15 '19 at 10:44
  • @konstunn that's exactly what I'm trying to figure out! Any suggestions? – datasplice May 12 '20 at 04:44
  • @datasplice, have a look here https://django-background-tasks.readthedocs.io/en/latest/#creating-and-registering-tasks. You can do `task(*args, schedule=1)` for example to schedule task at 1 second from now. Maybe one can do `schedule=0`, but I am not sure. – konstunn Jun 05 '20 at 04:50
  • @aboutaaron As of Oct 2020, I am using it with Python 3.6 and Django 3.1 – am70 Oct 25 '20 at 06:40
  • @konstunn the processes are ran from a job worker , that polls, so it may happen that a job will start up to 5 seconds after it is scheduled, – am70 Oct 25 '20 at 06:49
  • The problem is that you need to trigger `background-tasks` to trigger the scheduled tasks e.g in another terminal – CutePoison Mar 06 '21 at 09:38
7

If you're willing to install a 3rd party library, but you want something a whole lot simpler than Celery, check out Redis Queue. It does require Redis, which is pretty easy in itself, but that can provide a lot of other benefits as well.

RQ itself has almost zero configuration. It's startlingly simple.

References:

John Lehmann
  • 7,975
  • 4
  • 58
  • 71
  • 3
    This question is years old at this point, but I have now been using RQ for many such tasks. It really is startlingly simple and works like a charm. Highly recommended. – Marc Jan 31 '19 at 00:43
  • 1
    Not sure how this "accepted answer" answers the question "Isn't there a simple way to do this in Django without having to add (yet another) 3rd party package?", but the project looks neat indeed – Benjamin Toueg Aug 26 '21 at 18:03