How to make any method from view/model as celery task

Question

I have some of the analytics methods in models.py under class Analytics (e.g: Analytics.record_read_analytics()). And we are calling those methods for recording analytics, which doesn't need to be synchronous. Currently it's affecting rendering of each request so decided to add these methods in celery queue. We are already using celery for some of our tasks hence we have tasks.py and celery.py file.

Following is section of models.py file:

class Analytics():
    ...
    ...

    @staticmethod
    def method_a():
        ...
        ...


    def method_b():
        ...
        ...

    @staticmethod
    def record_read_analytics():
        ...
        ...

I don't wanted to write again same model level class methods in tasks.py and wanted to make some of view method's and model level class methods as celery task.

Following is celery.py file:

from __future__ import absolute_import
from celery import Celery

app = Celery('gnowsys_ndf',
             include=['gnowsys_ndf.tasks'])

app.config_from_object('gnowsys_ndf.celeryconfig')

if __name__ == '__main__':
    app.start()

I'm new to celery and looking for help. Thank you in advance.

http://stackoverflow.com/questions/9250317/using-class-methods-as-celery-tasks — khajvah, Sep 14 '16 at 12:14
Passing object arguments to celery function was giving an error but it was not visible to me so used [celery flower](http://flower.readthedocs.io/en/latest/) to dignose the problem and it pointed out correctly. Also thanks to @khajvah and Moinuddin for some useful corrections. Now it's working fine! — Kedar.Aitawdekar, Sep 14 '16 at 15:24

score 3 · Answer 1 · answered Sep 14 '16 at 12:20

You can create tasks out of methods. The bad thing about this is that the object itself gets passed around (because the state of the object in worker has to be same as the state of the caller) in order for it to be called, so you lose some flexibility. So your object has to be pickled every time, which is why I am against this solution. Of course this concerns only class methods, static methods have no such problem.

Another solution, which I like, is to create separate tasks.py or class based tasks and call the methods from within them. This way, you will have FULL control over Analytics object within your worker.

score 0 · Answer 2 · answered Sep 14 '16 at 12:18

0

You may achieve it like:

analytics = Analytics()   # Object of Analytics class
Analytics.record_read_analytics.delay()

Also, you need to add @task decorator with the record_read_analytics function

answered Sep 14 '16 at 12:18

Moinuddin Quadri

46,825
13
96
126

I tried this but task is not going to celery queue. Actually task is not getting register. – Kedar.Aitawdekar Sep 14 '16 at 12:34
check that your message broker service i running – Moinuddin Quadri Sep 14 '16 at 12:35
is the above code throwing some error or exiting without any action? – Moinuddin Quadri Sep 14 '16 at 12:36
2

Also check you have included your `models.py`, i.e. `include=['gnowsys_ndf.tasks', 'models.py']` – khajvah Sep 14 '16 at 12:37
Yes. I agree with khajvah. And you also need to restart the celery service in order to make these changer reflect with celery process – Moinuddin Quadri Sep 14 '16 at 12:40
Thank you @khajvah and Moinuddin for your help. These were small points to be corrected as well as some code or argument needs to be change. I fixed this and now it's working! – Kedar.Aitawdekar Sep 14 '16 at 15:21

How to make any method from view/model as celery task

2 Answers2