given a Redis Broker and a db-backend, where do Celery Task requests first get written to?

Question

Is there anyway to interrogate either redis, celery or postgres, to track a task request before its completion?

config:

BROKER_URL = "redis://localhost:6379"
CELERY_RESULT_BACKEND = "django-db"

from celery import Celery
from django.conf import settings

app = Celery(
   "someapp",
)

Now, after a while, the Table django_celery_results_taskresult has the tasks, with its id, task_id, task_name and result, among other things.

  id  |             task_name             | status  |           date_done
------+-----------------------------------+---------+-------------------------------
 2162 | someapp.pssystem.tasks.initialize | SUCCESS | 2019-07-16 16:55:41.101537-07
 2163 | someapp.pssecurity.tasks.flush    | SUCCESS | 2019-07-16 17:11:45.599822-07
 2164 | someapp.pssecurity.tasks.flush    | SUCCESS | 2019-07-16 17:18:49.798436-07
 2165 | someapp.pssecurity.tasks.flush    | SUCCESS | 2019-07-16 17:26:45.349578-07
 2166 | someapp.pssecurity.tasks.flush    | SUCCESS | 2019-07-16 17:31:49.27337-07

But that is only after the task has completed successfully or had an error. I am essentially launching long-running batches, which can easily take 2-3 minutes and I'd like to be able to acknowledge that the task was launched and poll its status before completion.

If I got to redis-cli, things aren't very clear there either, this looks it has more to do with celery workers than anything else:

127.0.0.1:6379> select 0
OK
127.0.0.1:6379> keys *celery*
1) "_kombu.binding.celery.pidbox"
2) "_kombu.binding.celeryev"
3) "_kombu.binding.celery"
127.0.0.1:6379> smembers "_kombu.binding.celeryev"
1) "worker.#\x06\x16\x06\x16celeryev.026d2bed-ebc8-4c8b-a8c1-732cd847b381"
2) "worker.#\x06\x16\x06\x16celeryev.2f6efd77-f931-4c01-b45f-000f3ab9f5bd"
3) "worker.#\x06\x16\x06\x16celeryev.c0da9ba7-31fd-4a6b-be62-685423c7b542"
4) "worker.#\x06\x16\x06\x16celeryev.09ece5fd-3a45-4912-9c0f-ea1f9e67c930"
5) "worker.#\x06\x16\x06\x16celeryev.ad784e1f-47ad-402f-8359-295c35138fba"
6) "worker.#\x06\x16\x06\x16celeryev.633275dc-a090-4c2f-9d18-1d6b2d00f8e5"
7) "worker.#\x06\x16\x06\x16celeryev.2ee3da5b-1d65-4f8c-b652-8907af7b6eb1"
8) "worker.#\x06\x16\x06\x16celeryev.617f6218-9382-4965-8d3e-c1ed70cf96e4"
9) "worker.#\x06\x16\x06\x16celeryev.64c9beef-2d35-418c-a562-b0a6f0c4054e"
127.0.0.1:6379> smembers "_kombu.binding.celery"
1) "celery\x06\x16\x06\x16celery"

versions:

postgres                  9.6
celery                    4.3.0
Django                    2.2.3
django-celery-results     1.0.4

Answer 1

Multiple ways to monitor tasks.

1. You can use flower to monitor the tasks. It's a realtime monitor and web admin for celery distributed task queue. https://github.com/mher/flower

2. Or You use celery API to fetch the task info.

   result = app.AsyncResult(task_id)
   

Answer 2

Its depends a little on what the intent is, but you can go a couple of different routes:

1. Signals - hooks that are called in different stages of the celery lifecycle. Example - log when a task completes
2. Inspection - An api built into celery to see status of different tasks. This is what flower uses under the hood
3. Redis API - Use the python Redis implementation to inspect a given queue. This is similar to what you were doing on the cli, but I have linked some examples

Based on your use case I think that a combo of signals to track starting and the redis api for polling active could work.

Answer 3

Upvoted both above, but what ultimately worked, to get rows in Table.django_celery_results_taskresult as soon as task starts was to change the Celery configuration:

credit goes to : Celery tasks, states and results

app = Celery("foo")
app.conf.update(task_track_started=True)  #  ✅