9

I am using Celery 3.1.16 broker (running RabbitMQ) and multiple Celery workers with celeryd daemonized through supervisor. Problem is with tasks update. When I update my tasks.py file, celery worker runs old code.

Celery launch command:

/home/my_project/bin/celery -B --autoreload --app=my_app.celery:app worker --loglevel=INFO

I include tasks file in django settings.py:

CELERY_IMPORTS = [
    'my_app.tasks'
]

pyinotify is installed and works (I guess so), part of celery logs:

[2014-12-16 20:56:00,016: INFO/MainProcess] Task my_app.tasks.periodic_update_task_statistic[175c2557-7c07-43c3-ac70-f4e115344134] succeeded in 0.00816309102811s: 'ok!'
[2014-12-16 20:56:11,157: INFO/MainProcess] Detected modified modules: ['my_app.tasks']
[2014-12-16 20:57:00,001: INFO/Beat] Scheduler: Sending due task my_app.tasks.periodic_update_task_statistic (my_app.tasks.periodic_update_task_statistic)
[2014-12-16 20:57:00,007: INFO/MainProcess] Received task: my_app.tasks.periodic_update_task_statistic[f22998a9-dcb4-4c29-8086-86dd6e57eae1]

So, my question: how to get celery update and apply new tasks code, if they were modified?

LJ Adams
  • 91
  • 1
  • 2
  • I don't have an answer. I still just reboot my workers when I do a code push. http://stackoverflow.com/questions/16929264/how-can-i-automatically-reload-tasks-modules-with-celery-daemon – pztrick Dec 16 '14 at 21:45
  • Same here. I restart celery at the end of the deployment procedure. Deployment is done using fabric for that matters. – Benoît Latinier Dec 16 '14 at 21:58
  • Hi LJ, any updates on this question? Were you able to figure out? – Desprit Oct 26 '15 at 22:03
  • 1
    I use supervisor in docker for local development and I also can't find out how to make --autoreload work. – kumar303 Dec 21 '15 at 18:02

3 Answers3

2

Celery has open issue with this problem https://github.com/celery/celery/issues/1025

Oleksandr
  • 306
  • 1
  • 8
1

I have this same problem. While I don't like it, I do the following, which first removes and compiled .pyc files anywhere under my current directory, then restarts all the workers. find . -name "*.pyc" -exec rm {} \; supervisorctl restart all It seems strange that the --autoreload flag does nothing, but it doesn't in my case.

DaveA
  • 1,227
  • 2
  • 14
  • 19
1

Celery only autoreloads those modules that it directly loaded, it does not keep an eye on other modules that loaded by the direct modules.

user2626972
  • 539
  • 2
  • 7
  • 14