I am using Flask with Celery & Celery beat.
I have a task that is running every 2 and 10 minutes. I need to store a global object that each task can access at any given time. Here's my celeryconfig
:
store = {"foo": 1, "bar": 0}
CELERYBEAT_SCHEDULE = {
"2-minutes": {
"task": "server.social.crawler.crawl",
"schedule": crontab(minute=f"*/2"),
"kwargs": dict(store=store),
},
"10-minutes": {
"task": "server.social.crawler.crawl",
"schedule": crontab(minute=f"*/10"),
"kwargs": dict(store=store),
}
}
I am looking for a way to change store
from within the crawl
task, so the next time it runs (after about 8 minutes), store
is up to date with the changes. Currently, when I change store
from within the task, it doesn't reflect in the next invocation of the task:
@celery.task()
def crawl(store: Dict[str, int]):
print(store) # always prints {"foo": 1, "bar": 0}
store["foo"] = 2