Why is Celery periodic task not working? (Django)

Question

It returns no errors, however when running celery -A mysite beat -l info & celery -A mysite worker -l info the task doesn't happen. When trying the same structure in another Django project it works fine. Help please!

My code:

mysite/settings.py (Only CELERY part)

BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Africa/Nairobi'

mysite/init.py

from __future__ import absolute_import
from .celery import app as celery_app

mysite/celery.py

from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')
app = Celery('mysite')

app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

personal/tasks.py

from celery.task.schedules import crontab
from celery.decorators import periodic_task
from celery.utils.log import get_task_logger

logger = get_task_logger(__name__)


@periodic_task(
    run_every=(crontab(minute='*/1')),
    name="task_deleteFiles",
    ignore_result=True
)
def task_deleteFiles():
    logger.info("Files Deleted")

PS: If necessary I can post the outputs from Celery

You have to set up a heartbeat - https://stackoverflow.com/questions/20957134/celery-heartbeat-not-working?rq=1 and https://stackoverflow.com/questions/42354064/django-celerybeat-periodic-task-only-runs-once — dmitryro, Jul 08 '18 at 16:45
@dmitryro I also thought so, however the task works completely fine in another project (simply by changing the names from the files). — Gustavo Machado, Jul 08 '18 at 17:01
A heartbeat is environment-specific (Docker, Vagrant, VM) and is **a process** that may not be running consistently in your current environment - you need to verify if this process is running for this specific environment and adjust the configuration accordingly. — dmitryro, Jul 08 '18 at 17:08
@dmitryro I completely understand that (even done some further research in the topic), the settings.py part is identical to another website of mine and it works fine (even without the heartbeat), somehow the other website works and this one doesn't. — Gustavo Machado, Jul 08 '18 at 17:19
Also the heartbeat is provided by the command: celery -A mysite beat -l info — Gustavo Machado, Jul 08 '18 at 17:24

score 1 · Accepted Answer · answered Jul 08 '18 at 18:35

1

Apparently by deleting app.py and app.pyc (which were under the personal file) celery works perfectly normal. Python was possibly trying to import somehow the module, strangely it didn't return any error messages.

answered Jul 08 '18 at 18:35

Gustavo Machado

51
7

score 0 · Answer 2 · answered Apr 19 '20 at 13:18

For others who find this page: If you are using Docker and you have changed celery beat settings during development, then there's a reasonable chance the container has old .pyc files that it is using as cache.

To avoid this error, delete all *.pyc files from your repo and add **/*.pyc to .dockerignore.

Why is Celery periodic task not working? (Django)

2 Answers2