run celery task only once on button click

Question

I created a django app with a celery task that organises database tables. The celery task takes about 8 seconds to do the job ( quite heavy table moves and lookups in over 1 million lines )

This celery starts through a button click by a registered user.

I created a task-view in views.py and a separate task-url in urls.py. When navigating to this task-url, the task starts.

My Problem:

When you refresh the task-url while the task is not finished, you fire up a new task.
With this set-up, you navigate to another page or reload the view

How do I prevent the task from firing up each time on refresh and prevent navigating to another page. Ideally, a user can only run one task of this particular type.

Might this be doable with JQuery/Ajax? Could some hero point me out in the right direction as I am not an expert an have no experience with JQuery/Ajax.

Thanks

score 2 · Accepted Answer · answered Nov 02 '17 at 13:24

"How do I prevent the task from firing up each time on refresh"

First make sure you use the "POST/redirect/GET" pattern: your view should only fire the task on a "POST" request (GET request must not have side-effects), and then return a HttpResponseRedirect.

This won't prevent the user from firing a second task while the first is still running, but it does prevent re-submitting the form each time you refresh ("GET") the page...

Ideally, a user can only run one task of this particular type.

When firing a task, store it's id in the session, and before firing a task check the session for a task id. If there's one, use it to check the task's state. If it's done (whatever the result), remove the task id and proceed with the new task, else just display a message telling your user he already has a task running.

Just make sure you correctly handle the case of a "ghost" task_id in the session (it might be a long finished task for which celery already discarded the results, or a task that got lost in a worker or broker crash - yeah, sh!t happens - etc). A working solution is to actually store a (timestamp, task_id) pair and if celery has no state for the task_id - actually "no state" is (mis)named celery.states.PENDING, which really means "I don't have a clue about this task's state ATM" - check the timestamp. If it's way older than it should then you can probably consider it as long dead and 6 feets under.

and prevent navigating to another page.

Why would you want to prevent your user to do something else in the meantime, actually ? From a UI/UX point of view, once the task is fired, your user should be redirected to a "please wait" page with (as much as possible) some progress bar or similar feedback. The simple (if a bit heavy) solution here is to do some polling using ajax : you setup a view that takes a task_id, check results / progress and returns them as json, and the 'please wait' page calls this views (using ajax) every X seconds to update itself (and possibly redirect the user to the next page when the task is done).

Now if there are some operations (apart from re-launching the same task) your user couldn't do while a task is running, you can use the same "check session for a current task" mechanism for those operations (making it a view decorator really helps).

Hi Bruno, I finally went for an Ajax POST to get the button registering towards the views.py and went for cache with 30 seconds timeout. having a specific timeout for one specific sessions variable would not work out. regarding the navigating to another page: that was only relevant when I was using a specific url toward a similar html. — radzia2, Nov 02 '17 at 20:13
one additional thing going on with my cache for the moment. I can not clear the cache from my celery task as this is the cache from the "worker", and not from the "web" proces. any idea on how to do this? or go for something else then cache? — radzia2, Nov 03 '17 at 12:45
Cache are fines, until you have to invalidate them . More seriously, if you're going with a cache-based solution, you do have to make this cache accessible from both Django and Celery processes - IOW you have to use a distributed cache. If you're using Redis as the results backend, then use it for caching too. Cannot help much more without detailed informations... — bruno desthuilliers, Nov 03 '17 at 12:57

score 0 · Answer 2 · answered Nov 02 '17 at 12:59

I believe it can be done more easily on the server side using locks,see this question Running "unique" tasks with celery

If it needs to be done on the client side, possible approach would be setting cookie like this

if (typeof $.cookie("running") === "undefined"){
  var date = new Date();
  var minutes = 20;
  date.setTime(date.getTime() + (minutes * 60 * 1000));
  $.cookie("running", "true", { expires: date });
}

and deleting when the task is finished

run celery task only once on button click

2 Answers2