4

I'm using Celery with RabbitMQ backend. How can I monitor and get the number of pending tasks? "celery events" and djcelery shows the running and completed tasks. Should I be monitoring RabbitMQ? If so how?

Ruggiero Spearman
  • 6,735
  • 5
  • 26
  • 37

1 Answers1

4

If you are using RabbitMq as broker, I recommend to use for that rabbitmqctl ( usually /usr/sbin/rabbitmqctl ). So command:

rabbitmqctl list_queues

Shows you number of tasks in queues. Other solution is installing rabbitmq-plugins: management plugin. You can do it with:

rabbitmq-plugins list
rabbitmq-plugins enable rabbitmq_management
service rabbitmq-server restart

Your rabbitmq-server will provide webagent where you can easily manage your rabbit-server.

Rustem
  • 2,884
  • 1
  • 17
  • 32
  • I did these. But according to Celery, each task result is put in a different queue in RabbitMQ. If I have thousands of tasks, RabbitMQ shows thousands of queues, each corresponding to one task. – Ruggiero Spearman Jul 15 '12 at 13:19
  • It's very specific installation. PLease check your celery config, I think its not correct – Rustem Jul 15 '12 at 13:20
  • 1
    It's possible I misinterpret it. Here it says: "Every new task creates a new queue on the server, with thousands of tasks the broker may be overloaded with queues and this will affect performance in negative ways." http://docs.celeryproject.org/en/latest/userguide/tasks.html#task-result-backends – Ruggiero Spearman Jul 15 '12 at 13:47
  • Is there a way to access this from celery? from a python script instead of cli – Sandeepa Kariyawasam Sep 23 '21 at 12:24