4

I'm wondering what is the difference between the Heroku Scheduler add-on and the Heroku Temporize Scheduler add-on. They both seem to be free and do scheduled jobs.

And how do they both compare to running a Python sched in Heroku?

I would like to run just one cron job to scrape some websites every minute with Heroku Python saving to Postgres. (I'm also trying to figure out what to write in a cron job to do so, but that's another question.)

Update with the solution:

Thanks to danneu's suggestions, the working solution was using the Heroku Scheduler. It was super simple to set up thanks to this tutorial.

(I tried using sched and twisted, but both times I got:

Application Error
An error occurred in the application and your page could not be served. Please try again in a few moments.

If you are the application owner, check your logs for details.

This was possibly due to my lack of experience of putting them in the correct place. It didn't work with a sync worker Heroku guricorn. I don't know the details.)

Community
  • 1
  • 1
Melissa
  • 1,236
  • 5
  • 12
  • 29

1 Answers1

2

Temporize is a 3rd party service. You'd have to read what the limitations of their free plan is. It looks like the free plan only lets you run a task 20 times per day which is a far cry from your needs of a 1-minute interval.

Heroku's Scheduler is offered by Heroku. It spins up a server for each task and bills you for the runtime. The minimum task interval on Heroku scheduler is 10 minutes which also won't give you what you want.

For a 1-minute interval that scrapes a page, I'd just run a concurrent loop in my Heroku app process. setTimeout in Javascript is an example of a concurrent loop that can run alongside your application server (if you were using Node). It looks like the Twisted example in your Python sched link is the Python equivalent.

If you're using Heroku's free-tier (which I think you are after seeing you in IRC), you get 1,000 runtime hours each month once you verify your account (else you only get 550 hours), which I imagine means giving them your credit card number. 1,000 hours is enough for a single dyno to run all month for free.

However, the free-tier dyno will sleep (turn off) if it has gone X amount of minutes without receiving an HTTP request. Obviously the Twisted/concurrent loop approach will only work while the dyno is awake and running since the loop runs inside your application process.

So if your dyno falls asleep, your concurrent loop will stop until your dyno wakes back up and resumes.

If you want your dyno to stay awake all month, you can upgrade your dyno for $7/mo.

danneu
  • 9,244
  • 3
  • 35
  • 63