3

I need to build a background job that goes through a list of RSS feeds and analyze them say every 10 minutes.

I have been using delayed_job for handling background jobs and I liked it a lot. I believe though that it's not built for recurring background jobs. I guess I can auto-schedule background job at the end of everyone (maybe with begin..rescue just to ensure it gets executes). Or preschedule say a month of advance worth of jobs and have another one that reschedule the every month..etc

This raised some concerned to me as I started asking myself: what if the server goes down in the middle of execution and the jobs didn't get scheduled?

I have also looked at Daemons gems which seemed the like it runs simple Ruby scripts with start/stop commands. I like the way delayed_job schedules and handles retries.

What do you recommend using in this case? What do you think the best way to design such a system with recurring background jobs? Also do you know a way I can monitor that background process and get notified if it stops?

Tam
  • 11,872
  • 19
  • 69
  • 119

3 Answers3

2

I just implemented delayed_job for a similar task (using :run_at => 2.days.from_now) and found it to be a perfect fit. The easiest way to handle your concern about a process failing is to make the first step of the job to create the next job. Also, you can create a has_many relationship to the delayed_job model which would allow you to access the :last_error. Or, look at the "Hooks" section of readme and it has a perfect example for failure.

Patrick Connor
  • 582
  • 5
  • 17
1

I think that this was a similar question: A cron job for rails: best practices? - not only are there answers, but also links to railscasts about background jobs in rails.

I used cron + delayed_job, but scheduled tasks were supposed to run few times a day, mostly just once.

Community
  • 1
  • 1
santuxus
  • 3,662
  • 1
  • 29
  • 35
0

Take a look at SimpleWorker. It's an elastic scheduling and background processing worker queue. It's cloud based and has persistence and redundancy so you don't need to worry if your servers go down or are restarted.

Very flexible in terms of scheduling, provides great introspection of jobs in the queue as well as notifications on status and errors.

Full disclosure: I work at SimpleWorker.

frommww
  • 129
  • 4