nodejs Kue job processing logic

Question

I have a very simple logical question.

I will be running job processing logic on a separate app server.

My job processing app will be a standalone app, doing nothing just processing jobs.

In my code, how do I make sure that my app continuously keep checking redis server for jobs ? -Do I need to run the code in infinite loop ? -or Do I need to keep restarting my app

or there is some inbuilt mechanism in Kue that I'm missing here ?

Thanks

This section answers your question - https://github.com/Automattic/kue#processing-jobs - you don't need to keep restarting your app, `queue.process` is called for every job in the queue — Alex, Jun 15 '15 at 15:52
Does it mean queue.process will keep checking the database for incoming jobs ? — pkpk, Jun 15 '15 at 15:59
or does it mean, whenever a new job is added to redis queue.process will be invoked ? — pkpk, Jun 15 '15 at 16:00
yes, as far as I know queue.process will be invoked when a job is added. — Alex, Jun 15 '15 at 16:01
okay, so that means any app with queue.process will live forever in memory after I start it ?, I will try it and update here ..Thx — pkpk, Jun 15 '15 at 16:06
This is what I did, my job producer, I ran it 5 times so that I have 5 jobs in my redis server. My job processing app (which is processing one job at a time) only able to fetch one job from redis server. I had to restart the job processing app multiple times to fetch all the jobs from Redis server. — pkpk, Jun 17 '15 at 01:56
Another thing I noted, if I start the job processing app first, it just sits idle and as soon as I publish a job from job producer, my job processing code executes. However, for the next job, it just sits there until I restart the job processing app. — pkpk, Jun 17 '15 at 02:02
okay, I was missing calling done () in the job processing callback. In the end Alex answer was correct. — pkpk, Jun 17 '15 at 03:05

score 2 · Accepted Answer · answered Jun 18 '15 at 11:37

See the documentation - https://github.com/Automattic/kue#processing-jobs

While there is a queue, it will continually run, and pick off jobs.

As per the example:

var kue = require('kue')
 , queue = kue.createQueue();

queue.process('email', function(job, done){
  email(job.data.to, done);
});

function email(address, done) {
  if(!isValidEmail(address)) {
    //done('invalid to address') is possible but discouraged
    return done(new Error('invalid to address'));
  }
  // email send stuff...
  done();
}

nodejs Kue job processing logic

1 Answers1