Stack Overflow
Stack Overflow

Questions tagged [celerybeat]

The Celery periodic task scheduler

Celery beat is a scheduler for periodic tasks in Celery. It provides useful functions to create and execute tasks on certain intervals, with support for fail-over, retries, status tracking etc. For more info look at project documentation: http://celery.github.com/celery/internals/reference/celery.beat.html

514 questions
62
votes
9 answers

How to dynamically add / remove periodic tasks to Celery (celerybeat)

If I have a function defined as follows: def add(x,y): return x+y Is there a way to dynamically add this function as a celery PeriodicTask and kick it off at runtime? I'd like to be able to do something like (pseudocode): some_unique_task_id =…
Jamie Forrest
  • 10,895
  • 6
  • 51
  • 68
32
votes
3 answers

Disable pidfile for celerybeat

celeryd doesn't require a pidfile, but celerybeat seems to. Is there any way to disable it? I'm using Upstart to manage processes so using a pidfile is redundant.
bradley.ayers
  • 37,165
  • 14
  • 93
  • 99
30
votes
10 answers

on building docker image level=error msg="Can't close tar writer: io: read/write on closed pipe"

time="2017-10-27T07:39:20Z" level=error msg="Can't add file /var/app/current/app/content_classifier/forest.pickle to tar: io: read/write on closed pipe" time="2017-10-27T07:39:20Z" level=error msg="Can't close tar writer: io: read/write on closed…
Aniket patel
  • 551
  • 1
  • 5
  • 17
30
votes
2 answers

In celery 3.1, making django periodic task

Things changed too much in Django, so I can't use 3.1. I need some help. I read about make a task in django, and read Periodic Tasks document. But I don't know how make periodic tasks in django. I think this becuase of my low level English.. In the…
margincall
  • 483
  • 1
  • 6
  • 24
26
votes
3 answers

Celery stop execution of a chain

I have a check_orders task that's executed periodically. It makes a group of tasks so that I can time how long executing the tasks took, and perform something when they're all done (this is the purpose of res.join [1] and grouped_subs) The tasks…
Salami
  • 2,849
  • 4
  • 24
  • 33
25
votes
6 answers

Connect new celery periodic task in django

It's not a question but help to those who will find that the declaration of periodic tasks described in celery 4.0.1 documentation is hard to integrate in django: http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html#entries copy…
vvkuznetsov
  • 1,056
  • 1
  • 9
  • 16
22
votes
4 answers

Running celery worker + beat in the same container

My flask app is comprised of four containers: web app, postgres, rabbitMQ and Celery. Since I have celery tasks that run periodically, I am using celery beat. I've configured my docker-compose file like this: version: '2' services: rabbit: #…
hugo
  • 1,175
  • 1
  • 11
  • 25
20
votes
8 answers

Check if celery beat is up and running

In my Django project, I use Celery and Rabbitmq to run tasks in background. I am using celery beat scheduler to run periodic tasks. How can i check if celery beat is up and running, programmatically?
shinoymm
  • 315
  • 1
  • 3
  • 10
20
votes
1 answer

Celery : Execute task after a specific time gap

I want to send an email to my users exactly 48 hours after they have registered.How do i achieve this using celery? If I create a periodic task to send an email, I will have to decide a specific time during which i want to execute that task. I don't…
mukesh
  • 726
  • 1
  • 11
  • 27
18
votes
1 answer

Work around celerybeat being a single point of failure

I'm looking for recommended solution to work around celerybeat being a single point of failure for celery/rabbitmq deployment. I didn't find anything that made sense so far, by searching the web. In my case, once a day timed scheduler kicks off a…
Dmitry Grinberg
  • 695
  • 2
  • 7
  • 15
18
votes
2 answers

Celerybeat not executing periodic tasks

How do you diagnose why manage.py celerybeat won't execute any tasks? I'm running celerybeat via supervisord with the command: /usr/local/myapp/src/manage.py celerybeat --schedule=/tmp/celerybeat-schedule-myapp --pidfile=/tmp/celerybeat-myapp.pid…
Cerin
  • 60,957
  • 96
  • 316
  • 522
17
votes
4 answers

Celery beat not picking up periodic tasks

I am trying to get started with celery, but I can't get my task up and running. I have installed django-celery-beat and celery4. My settings file. Installed apps (with celery packages) ... 'django_celery_beat', 'django_celery_results' the celery…
user5170375
15
votes
2 answers

Celery worker and beat load in one command

Is there a way to start the celery worker and beat in one command? I would like to add celery to my automated deployment procedure with Fabric. I am currently running: celery -A prj worker -B followed by celery -A prj beat -l info -S…
zubhav
  • 1,519
  • 1
  • 13
  • 19
15
votes
3 answers

celery beat schedule: run task instantly when start celery beat?

If I create a celery beat schedule, using timedelta(days=1), the first task will be carried out after 24 hours, quote celery beat documentation: Using a timedelta for the schedule means the task will be sent in 30 second intervals (the first task…
timfeirg
  • 1,426
  • 18
  • 37
14
votes
2 answers

Django/Celery multiple queues on localhost - routing not working

I followed celery docs to define 2 queues on my dev machine. My celery settings: CELERY_ALWAYS_EAGER = True CELERY_TASK_RESULT_EXPIRES = 60 # 1 mins CELERYD_CONCURRENCY = 2 CELERYD_MAX_TASKS_PER_CHILD = 4 CELERYD_PREFETCH_MULTIPLIER =…
Neara
  • 3,693
  • 7
  • 29
  • 40
1
2 3
…
34 35