1

I wanted to perform some data processing in parallel using Bull NPM and start processing each job at the given cron Time

const Queue = require("bull"),
        
     /**
     *  initialize the queue for executing cron jobs
     **/
    
     this.workQueue = new Queue(this.queueOptions.name, {
          redis: redisConfig
     });
        
        
         this.workQueue.process((job, done) => {
              done();
              this.processJob(job)
                .then(data => {
                  global.logger.info(`successfully-executed-job ${job.id}`);
                })
                .catch(error => {
                  global.logger.error(`JSON.stringify(error)}-in-executing-job-${job.id}`); 
                });
            });
        
    // here I have included Unique JobId     
        
    this.workQueue.add({}, {repeat: { cron:"5 * * * *",jobId:Date.now()});

Any suggestions to achieve the same?

Sagar Pednekar
  • 334
  • 4
  • 14

2 Answers2

0

The issue is resolved now if you're facing the same issue make sure that you're referring to the correct timezone.

Cheers!!

Sagar Pednekar
  • 334
  • 4
  • 14
0

I also faced this same issue. One thing to note with respect to the above code is that a Queuescheduler instance is not initialized. Ofcourse timezone also plays a crucial role. But without a Queuescheduler instance (which has the same name as the Queue), the jobs doesnt get added into the queue. The Queuescheduler instance acts as a book keeper. Also take care about one more important parameter "limit". If you dont set the limit to 1, then the job which is scheduled at a particular time will get triggered unlimited number of times.

For example: To run a job at german time 22:30 every day the configuration would look like:

    repeat: { 
        cron: '* 30 22 * * *',
        offset: datetime.getTimezoneOffset(),
        tz: 'Europe/Berlin',
        limit: 1
    }

Reference: https://docs.bullmq.io/guide/queuescheduler In this above link, the documentation clearly mentions that the queuescheduler instance does the book keeping of the jobs.

In this link - https://docs.bullmq.io/guide/jobs/repeatable, the documentation specifically warns us to ensure that we instantiate a Queuescheduler instance.

Dharman
  • 30,962
  • 25
  • 85
  • 135