3

I'm using the django celery task queue, and it works fine in development, but not at all in wsgi production. Even more frustrating, it used to work in production, but I somehow broke it.

"sudo rabbitmqctl status" tells me that the rabbitmq server is working. Everything also seems peachy in django: objects are created, and routed to the task manager without problems. But then their status just stays as "queued" indefinitely. The way I've written my code, they should switch to "error" or "ready," as soon as anything gets returned from the celery task. So I assume there's something wrong with the queue.

Two related questions:

  • Any ideas what the problem might be?
  • How do I debug celery? Outside of the manage.py celeryd command, I'm not sure how to peer into its inner workings. Are there log files or something I can use?

Thanks!

PS - I've seen this question, but he seems to want to run celery from manage.py, not wsgi.

Community
  • 1
  • 1
Abe
  • 22,738
  • 26
  • 82
  • 111

2 Answers2

2

After much searching, the most complete answer I found for this question is here. These directions flesh out the skimpy official directions for daemonizing celeryd. I'll copy the gist here, but you should follow the link, because Michael has explained some parts in more detail.

The main idea is that you need scripts in three places:

  1. /etc/init.d/celeryd
  2. /etc/default/celeryd
  3. myApp/settings.py

Settings.py appears to be the same as in development mode. So if that's already set up, there are four steps to shifting to production:

  1. Download the daemon script since it's not included in the installation: https://github.com/celery/celery/tree/3.0/extra/generic-init.d/
  2. Put it in /etc/init.d/celeryd
  3. Make a file in /etc/default/celeryd, and put the variables here into it: http://docs.celeryproject.org/en/latest/tutorials/daemonizing.html#example-django-configuration
  4. Start the script

This solved my problem.

David Drabik
  • 5
  • 2
Abe
  • 22,738
  • 26
  • 82
  • 111
1

I think the reason you are not getting any response from celery, is because celeryd server might not be running. You could find out about it by doing ps -ef |grep celeryd. In order to figure out what is the error while trying to run celeryd, you might want to do the following.

In your settings.py file you could give the path to the celery log file CELERYD_LOG_FILE = <Path to the log file> and while running celeryd server you could specify the level manage.py celeryd -l DEBUG.

Ritesh Kadmawala
  • 743
  • 9
  • 17
  • Thanks for the answer. The problem mysteriously fixed itself over the weekend. I'll post updates here if I learn anything new about the problem. – Abe Dec 12 '11 at 22:12
  • So, same problem popping up again. Using the ps | grep command you recommended, it's clear that the celery daemon *isn't* running. How do I start it? Dumb question maybe, but I can't find clear instructions. Do I just run nohup manage.py celeryd? – Abe Dec 13 '11 at 04:52