Can't see my celery logs when running beat

Question

I am starting celery via supervisord, see the entry below.

[program:celery]
user = foobar
autostart = true
autorestart = true
directory = /opt/src/slicephone/cloud
command = /opt/virtenvs/django_slice/bin/celery beat --app=cloud -l DEBUG -s /home/foobar/run/celerybeat-schedule --pidfile=/home/foobar/run/celerybeat.pid
priority = 100
stdout_logfile_backups = 0
stderr_logfile_backups = 0
stdout_logfile_maxbytes = 10MB
stderr_logfile_maxbytes = 10MB
stdout_logfile = /opt/logs/celery.stdout.log
stderr_logfile = /opt/logs/celery.stderr.log

pip freeze | grep celery

celery==3.1.0

But any usage of:

@celery.task
def test_rabbit_running():
    import logging
    from celery.utils.log import get_task_logger
    logger = get_task_logger(__name__)
    logger.setLevel(logging.DEBUG)
    logger.info("foobar")

doesn't show up in the logs. Instead I get entries like the following.

celery.stdout.log

celery beat v3.1.0 (Cipater) is starting.
__    -    ... __   -        _
Configuration ->
    . broker -> redis://localhost:6379//
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> /home/foobar/run/celerybeat-schedule
    . logfile -> [stderr]@%DEBUG
    . maxinterval -> now (0s)

celery.stderr.log

[2013-11-12 05:42:39,539: DEBUG/MainProcess] beat: Waking up in 2.00 seconds.
INFO Scheduler: Sending due task test_rabbit_running (retail.tasks.test_rabbit_running)
[2013-11-12 05:42:41,547: INFO/MainProcess] Scheduler: Sending due task test_rabbit_running (retail.tasks.test_rabbit_running)
DEBUG retail.tasks.test_rabbit_running sent. id->34268340-6ffd-44d0-8e61-475a83ab3481
[2013-11-12 05:42:41,550: DEBUG/MainProcess] retail.tasks.test_rabbit_running sent. id->34268340-6ffd-44d0-8e61-475a83ab3481
DEBUG beat: Waking up in 6.00 seconds.

What do I have to do to make my logging calls appear in the log files?

what is the contents of both `stdout_logfile` and `stderr_logfile`? — podshumok, Nov 12 '13 at 05:37

score 11 · Accepted Answer · edited May 23 '17 at 11:54

11

It doesn't log anything because it doesn't execute any tasks (and it's ok).

See also Celerybeat not executing periodic tasks

edited May 23 '17 at 11:54

Community

1
1

answered Nov 12 '13 at 22:02

podshumok

1,649
16
20

1

I've changed it to "celery worker --beat .." according to the new docs http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html#starting-the-scheduler and it works now, ty! – kev Nov 12 '13 at 22:54

score 0 · Answer 2 · edited May 23 '17 at 10:31

0

I'd try to put the call to log inside a task as the name of the util function implies get_task_logger, or just start with a simple print, or have your own log set up as suggested in Django Celery Logging Best Practice (best way to go IMO)

edited May 23 '17 at 10:31

Community

1
1

answered Nov 12 '13 at 05:27

Guy Gavriely

11,228
6
27
42

Ah sorry, I've updated my example. The code is inside a task which is listed in CELERYBEAT_SCHEDULE. I have tried the solution in "Django Celery Logging Best Practice" (added the entries in django's config file) but I get the same results I can just see the task xy sent messages but not my logging output. – kev Nov 12 '13 at 05:40

Can't see my celery logs when running beat

2 Answers2