1

I have celery task which needs to start an xmpp when the worker is started but it doesnt really work

from celery import Celery
from MyXmpp import MyXmpp

celery = Celery('myxmpp')
celery.config_from_object('celeryconfig')

myjabber = MyXmpp()

@celery.task 
def worker_send_jabber(message):
   myjabber.send_admin_xmpp(message)

If I do it like that it only starts the xmpp but not the worker/task. How can I get celery to initialize my xmpp and then send the message through that. I dont want it to connect and disconnect all the time. the xmpp client is supposed to be online as long as the worker is running.

dave
  • 161
  • 2
  • 12
  • 1
    similar issue but it doesnt work ... http://stackoverflow.com/questions/8750043/shared-xmpp-connection-between-celery-workers that is actually what I tried to do above but it only starts the xmpp but doesnt run the task itself. – dave Nov 29 '12 at 14:52

1 Answers1

3

There are several ways to do this, and a common way is:

_client = None

def get_client():
    global _client
    if _client is None:
        _client = MyXmpp()
    return _client

@celery.task
def send_jabber(message):
    get_client().send_admin_xmpp(message)

but some may like this version better:

from celery import Celery, Task
from celery.utils import cached_property

class XmppTask(Task):
    Client = MyXmpp
    abstract = True

    def __call__(self, *args, **kwargs):
        # passes self argument to the task body
        return super(XmppTask, self).__call__(self, *args, **kwargs)

    @cached_property
    def client(self):
        return self.Client()


celery = Celery()
@celery.task(base=XmppTask):
def send_jabber(self, message):
    return self.client.send_admin_xmpp(message)
asksol
  • 19,129
  • 5
  • 61
  • 68
  • btw, the second version will not cache connections between tasks inheriting from XmppTask, so this is a shortcoming from the first example. An even better way could be to use a pool of connections. – asksol Nov 30 '12 at 11:46
  • I will try this out but at the end I dropped celery on this today and wrote my on lib handling the message from the rabbitmq. I deliver them with django and celery.send_task to the queue and pick them up with my own handler in the my jabber client. this is working fine for me the connection doesnt need to be shared btw tasks there will always be only one jabber connection running and delivering them will take as long as it takes. but... I will try your solution anyway and will let you know if it is working. I will accept your solution as anwser anyway! thanks dave – dave Nov 30 '12 at 16:43