35

Using node-celery, we can enable node to push Celery jobs to the task queue. How can we allow node to be a Celery worker and consume the queue?

Nyxynyx
  • 61,411
  • 155
  • 482
  • 830
  • 2
    I am looking for the same solution. Can you let me know if you've found anything? – Natus Drew Jul 13 '14 at 21:04
  • @nathan Nothing yet. You can try upvoting the question! – Nyxynyx Jul 14 '14 at 00:53
  • @Nyxynyx Did you found a best way to achieve this? – Swamy Dec 15 '17 at 06:38
  • Celery is a Python task queue library. The "task" is a python function. So it makes no sense to talk about a node-js Celery worker. If you are using Node-js both ends you don't need to use Celery at all, use something native Javascript – Anentropic Aug 13 '18 at 17:39
  • @Anentropic totally wrong. It actually makes lots of sense, especially if you want to code your Celery tasks in JavaScript and in general have polyglot team. The whole Celery infrastructure is relatively simple and it should not be a problem to implement this. – DejanLekic Sep 29 '19 at 16:51

2 Answers2

8

For Celery if the end point is amqp. Checkout Celery.js Github any node process started as amqp consumer would work fine. For every other self.conf.backend_type types you can have varied consumer. Following example is merely for amqp.

One such example. The message below may be the Celery task object.

var amqp = require('amqp');
var connection = amqp.createConnection({ host: "localhost", port: 5672 });
connection.on('ready', function () {
  connection.queue("my_celery_queue", function(queue){
    queue.bind('#'); 
    queue.subscribe(function (message) {
      //eat your Celery work here
    })
  })
})
bhantol
  • 9,368
  • 7
  • 44
  • 81
1

Here is an approach from the Celery doc, by exposing REST api:

http://docs.celeryproject.org/en/latest/faq.html#is-celery-multilingual

Also, there’s another way to be language-independent, and that’s to use REST tasks, instead of your tasks being functions, they’re URLs. With this information you can even create simple web servers that enable preloading of code. Simply expose an endpoint that performs an operation, and create a task that just performs an HTTP request to that endpoint.

some example: http://ask.github.io/celery/cookbook/remote-tasks.html

Brankodd
  • 831
  • 9
  • 21