Guidelines for Scaling on Heroku Platform

Question

I'm running a NodeJS based application, connected to mongohq-MongoDB, on a free Dyno. I'd like to migrate it to use the hobby Dyno and the motivation to do so is not only avoiding sleep-time, but also to enable higher HTTP traffic throughput.

Reading the Scaling documentation and the Procfile article got me confused about how scaling on Heroku should be done.

In the Procfile article, it is said that the web process type is the only process which will receive HTTP traffic from Heroku routing-mesh.

So my questions are:

1. When there's already one hobby Dyno running, executing 'heroku ps:scale web+2' will result in having +2 web processes on the same Dyno or in adding two hobby Dynos (to a total of three hobby Dynos) ?
   • Total of three hobby Dynos means 3 web processes and 27 non-web processes available ?
2. In this answer, it is suggested to use the cluster module to fork threads to handle HTTP requests,How one can determine the # of workers that should be created (in the // fork worker processes loop)?
3. How should I decide when to scale my application horizontally (add more Dynos of the same type) of vertically (stronger Dynos like standard-1X/2X)
   • Horizontal scale should be triggered to handle higher # of requests?
   • Vertical scale should be triggered to handle heavier processing (more computational resources are in need to provide response to respective request?)

N.B,

In the Scaling section of this answer, the result Dynos is still not clear in respect to question #1 above.

---

Please take into consideration that application optimization (e.g. find/remove bottlenecks, etc...) is out of the scope of this question as the goal of it is to better our understanding of resource-utilization on Heroku platform.

Answer 1

I'll take these in order...

1. No, you have 10 process types in the Hobby tier, this means if you have 1 web dyno, you can have a further 9 dynos running for the same app, these further 9 can be of any entry in your Procfile.

   You can only have one web dyno, the remainder are the other (up to 9) entries in your Procfile.

2. That's mostly dictated by memory, it depends on what you're doing in your app, if it's fairly small, then you can run more, Node.js cluster workers are separate Node.js processes, that happen to share the same socket, and the OS distributes requests across processes.

3. You need to consider response times for requests here, if you're seeing "slow" requests, you either optimise, or when you're sure you're optimised, you need to scale up, this may or may not involve increasing the number of running dynos, it could be things that your app talks to that are slowing requests (e.g. databases, or external services).

What defines a "slow" request?

Well...you might consider a "budget" for response time, but you need to measure request/response time, often to a fairly high-level of granularity, so you can isolate what's causing the slow response, and ensure that you scale the right part, doubling your dyno count will not achieve anything if it's a poorly optimised database query that's causing you grief, if anything it could make things worse, so you need to measure the overall response time, and magic requests.

Horizontal scale and vertical scaling are quite different, and the answer to this depends on your app, larger dynos have more memory, which means that they can perhaps cache data in memory, or process bigger payloads from external services, also Performance-M and Performance-I dynos are "dedicated" to the customer, so, you will benefit from more predictable load profiles.

If you have multiple web dynos, then the Heroku router will distribute incoming requests across them randomly, meaning that they'll receive approximately the same number of requests in a period, and "Little's Law" applies here, there's a good explanation of how to apply that to web requests here, if your desired concurrent requests, at the current average response time doesn't work, you have two choices, reduce the average response time, or increase your capacity.

Also, the Hobby tier doesn't make your dynos faster, it allows you to have more processes (dynos) and they can run 24hrs a day, but you would need to go to one of the larger dyno types to get a performance boost.