1

I'm planning on deploying a dynamic site that needs certain tasks to be done periodically in the background, let's say every hour or two. The data that i need to output is strictly depending on the result of those queries.

Now, the problem is that those tasks require a certain amount of time, e.g. 8-10 minutes: so if a user asks for a page, and therefore a certain view gets called, and that task is still in process everything goes bad.

So, here's the question: what do i need to implement to handle that problem? I've been searching and i found about celery but i haven't understood how it could be used to fix the above issue and maybe implement a caching system that intervenes when the tasks aren't finished yet.

Nelloverflow
  • 1,511
  • 1
  • 11
  • 15

1 Answers1

2

A solution can be the following: The view will serve cached results, when a task is completed this one will update the cached results with the fresh ones. You can also use celery in this way here, when a cached view is hit and it has to be recalculated, a celery task get detached (the view will return the cached value).

Here there's an interesting example about a one-at-time celery task: http://ask.github.com/celery/cookbook/tasks.html

Tommaso Barbugli
  • 11,781
  • 2
  • 42
  • 41
  • Thanks for the answer. I went searching on how to update the cache results from Celery and saw it was a little bit of trouble. I found couple minutes ago this syfou.bitbucket.org/django-groupcache which i saw mentioned here stackoverflow.com/questions/2268417/… and it seems to do the trick. I tried it right now and it works. I guess i will put the whole thing under some test to stress it out and then integrate it with a Celery task. – Nelloverflow Sep 13 '11 at 09:02