15

If I create a celery beat schedule, using timedelta(days=1), the first task will be carried out after 24 hours, quote celery beat documentation:

Using a timedelta for the schedule means the task will be sent in 30 second intervals (the first task will be sent 30 seconds after celery beat starts, and then every 30 seconds after the last run).

But the fact is that in a lot of situations it's actually important that the the scheduler run the task at launch, But I didn't find an option that allows me to run the task immediately after celery starts, am I not reading carefully, or is celery missing this feature?

timfeirg
  • 1,426
  • 18
  • 37

3 Answers3

4

I decide I could just declare a instance of every task, and execute them at celery launch. I don't like this at all because it makes starting celery beat extremely slow (if you have slow PeriodicTask), but it does what I want.

simply add this to the end of tasks.py:

########### spawn all tasks at launch! ############
localmess = locals().values()
for obj in localmess:
    if isclass(obj):
        if obj is not PeriodicTask and issubclass(obj, PeriodicTask):
            instance = obj()
            logger.info('running {0}'.format(obj))
            try:
                instance.run()
            except:
                logger.warn('task fail: {0}'.format(obj))
                pass

######## all tasks must be decleared above! ########
timfeirg
  • 1,426
  • 18
  • 37
  • What if you just scheduled the tasks to be run 'now' (or a minute after if needed), so it won't block the beat startup. And what about iterating `locals()` directly, removing the unneccessary `isclass` check and the `pass` – smido Nov 13 '19 at 07:47
3

You can run tasks right when the worker is ready with the worker_ready.connect decorator :

from celery.signals import worker_ready

@worker_ready.connect
def at_start(sender, **kwargs):
    """Run tasks at startup"""
    with sender.app.connection() as conn:
        sender.app.send_task("app.module.task", connection=conn)

Credits go to this answer : https://stackoverflow.com/a/14589445/3922534

Mathieu Rollet
  • 2,016
  • 2
  • 18
  • 31
-1

The best idea is create an implementation which schedules the task itself after completing the task. Also, create an entrance lock so the task cannot be executed multiple times per moment. Trigger the execution once.

In this case,

  • you don't need a celerybeat process
  • the task is guaranteed to execute
Vitaly Greck
  • 658
  • 5
  • 9
  • 2
    that requires quite a lot implementation within each task, which I tend to avoid, because the tasks could gone wrong and fail to schedule the next one, I would have to handle exceptions in every task – timfeirg Jul 18 '15 at 04:18