node.js cluster module not helping how I thought it would

Question

I recently learned about the cluster module in node, and wanted to give it a spin. I have a super basic express app that look like this.

const express = require('express');
const app = express();
const cluster = require('cluster');
const os = require('os');

const numCores = os.cpus().length;

app.get('/', (req, res) => {
    res.status(200).json({ success: true })
});

if (cluster.isMaster) {
    for (let i = 0; i < numCores; i++) {
        cluster.fork();
    }

    cluster.on('exit', (worker, code, signal) => {
        console.log(`worker ${worker.process.pid} died`);
        console.log("Let's fork another worker!");
        cluster.fork();
    });

} else {
    app.listen(1337, () => console.log('server is running on port 1337'));
}

I am also using the Artillery load testing tool to load test this basic node app.

The problem is that whether I use the cluster module or not, by the time my load test is done running, I have the same amount of failed requests.

This is the output of my test

Scenarios launched: 30000 Scenarios completed: 23145 Requests completed: 23145 Mean response/sec: 426.14 Response time (msec): min: 0 max: 449 median: 1 p95: 4 p99: 35 Scenario counts: simple request: 30000 (100%) Codes: 200: 23145 Errors: ETIMEDOUT: 6855

Why do I have 6855 ETIMEDOUT errors even when I am using the cluster module? This is the exact same amount of errors I have when not using the cluster module.

Have you tried printing the process id in the middleware? See if they are actually being handled by different processes. Are you also using the server as a client to perform the stress test? — MinusFour, Oct 26 '21 at 15:47
I just tried printing the process id and each request has a different pid. I am running the test from the CLI — Chaim Friedman, Oct 26 '21 at 16:15
based on this answer it seems that this is expected behavior because node is single threaded read here: https://stackoverflow.com/a/28737438/6356919 — Chaim Friedman, Oct 26 '21 at 17:09
That's not exactly what the answer is saying though. You should expect some performance gains, albeit small, depending on how many cores you have. The question is also barely doing anything CPU bound and even then he did get a performance boost. — MinusFour, Oct 26 '21 at 17:55
Correct, and as you will see, I tried to implement the answer below, and while it did improve performance a bit, it was honestly negligible. This is why I think the linked answer in my comment answer my question as well — Chaim Friedman, Oct 26 '21 at 18:47
yup, seems to be the right answer, you hit the bottleneck somewhere else. In this scenario, if you reduce the number of workers to 2, you could still get similar result I think. Or, try increasing the load, you could see higher CPU utilization. — seanplwong, Oct 28 '21 at 16:22

seanplwong · Answer 1 · 2021-10-28T16:14:59.623

You probably just need to define handler inside the child:

const express = require('express');
const cluster = require('cluster');
const os = require('os');

const numCores = os.cpus().length;

if (cluster.isMaster) {
    for (let i = 0; i < numCores; i++) {
        cluster.fork();
    }

    cluster.on('exit', (worker, code, signal) => {
        console.log(`worker ${worker.process.pid} died`);
        console.log("Let's fork another worker!");
        cluster.fork();
    });
} else {
  const app = express();
  app.get('/', (req, res) => {
    let result = -1;
    console.time();
    for (let i = 0; i < 2 ** 28; i += 1) {
      result += i;
    }
    console.timeEnd();
    res.status(200).json({ result });
  });
  app.listen(1337, () => console.log('server is running on port 1337'));
}

I would put the const app = express() inside as well.

I think defining handler before you fork cause the problem.

SEE: Using cluster in an Expressjs app

I think the situation is your server implementation's bottleneck is not the processing power, it's somewhere else like IO or memory.

If you add some logic in the handler (see the script updated above), when there is only 1 worker, the CPU utilization will max out at about 25% if you have a quad core machine.

I don't have Artillery so I use the following script to test locally.

const fetch = require('node-fetch');

const CONCURRENT_REQUEST = 8;
let numberOfRequest = 100 - CONCURRENT_REQUEST;
let failed = 0;

const ping = () => (
  fetch('http://localhost:1337')
    .then(r => r.json())
    .then(({ success }) => {
      if (!success) {
        throw new Error('oops');
      }
    })
    .catch((e) => {
      // console.error(`index ${i}`, e);
      failed += 1;
    })
    .then(() => {
      numberOfRequest -= 1;
      if (numberOfRequest > 0) {
        return ping();
      }
    })
);

const requests = new Array(CONCURRENT_REQUEST).fill(0).map((_, i) => (
  ping()
));

Promise.all(requests).then(() => console.log('done. failed: ', failed));

With only 1 worker:

Minutes           : 1
Seconds           : 1
Milliseconds      : 199
TotalDays         : 0.000708324185185185
TotalHours        : 0.0169997804444444
TotalMinutes      : 1.01998682666667
TotalSeconds      : 61.1992096
TotalMilliseconds : 61199.2096

With 4 workers:

Seconds           : 17
Milliseconds      : 249
Ticks             : 172490478
TotalDays         : 0.000199641756944444
TotalHours        : 0.00479140216666667
TotalMinutes      : 0.28748413
TotalSeconds      : 17.2490478
TotalMilliseconds : 17249.0478

I tried this and added the `const app = express()` in the worker, and this helped a bit, my total errors was now down to 5609 from 6855. I still feel like this isnt right — Chaim Friedman, Oct 26 '21 at 16:25

node.js cluster module not helping how I thought it would

1 Answers1