10

Using Node.JS and cluster module.

I am trying to understand how multiple forked child processes can listen on the same port.

For example, using cluster module we can do this:

const port = 443;
...
if (cluster.isMaster) {
    for(let i = 0; i < numCPUs; i++)
    {
        cluster.fork();
    }
...
}
else // Forked child processes:
{
... 
    https.createServer({
        key: fs.readFileSync('server.key'),
        cert: fs.readFileSync('server.cert')
    }, app)
    .listen(port, () => {
        console.log(`HTTPS Listening on port ${port}`);
    });

}

This code forks multiple processes, all of which call listen on the same port. I'm not clear on how all the processes could bind the same port and still be able to determine which process gets the port traffic. Is this actually an illusion, and instead the master process is actually the only one binding the port and passing the requests randomly to the forked children? (If this is the case, isn't there a performance hit?)

Thanks for helping me understand how all the child processes can listen on same port at same time.

NOTE that this example is running on a Windows machine, but if I understand correctly, it is compatible with both Windows and Linux.

Ryan Griggs
  • 2,457
  • 2
  • 35
  • 58

1 Answers1

2

From the docs Cluster: How It Works:

The cluster module supports two methods of distributing incoming connections.

The first one (and the default one on all platforms except Windows), is the round-robin approach, where the master process listens on a port, accepts new connections and distributes them across the workers in a round-robin fashion, with some built-in smarts to avoid overloading a worker process.

The second approach is where the master process creates the listen socket and sends it to interested workers. The workers then accept incoming connections directly.

The "top level" process is the one binding the port. The IPC channel automatically distributes to the child processes. "Workers can share the TCP connection".

Another important part is the exclusive property of server.listen().

If exclusive is false (default), then cluster workers will use the same underlying handle, allowing connection handling duties to be shared. When exclusive is true, the handle is not shared, and attempted port sharing results in an error. An example which listens on an exclusive port is shown below.

So you could have them all try (and fail) to bind to the same port if you tell them to be exclusive, but by default (which is what is in your example), they share the handle allowing the connection to get distributed.

zero298
  • 25,467
  • 10
  • 75
  • 100
  • 2
    I'm still trying to figure out how cluster workers share the same handle. I know they do, I just don't see where it is in the code. The initialization code for a cluster as shown in the OP's question shows ONLY a the workers calling `.listen()` which is opposite of master listening and distributing. How does that come to be? – jfriend00 Dec 16 '19 at 17:33
  • @jfriend00 I was kind of trying to figure that out as well. I was looking at how [`node/lib/internal/cluster/master.js`](https://github.com/nodejs/node/blob/0646eda4fc0affb98e13c30acb522e63b7fd6dde/lib/internal/cluster/master.js#L292) creates a `RoundRobinHandle` in `queryServer`. It looks like there are special `internalMessage` that trigger the special handling and cause the cluster to use the shared connection. – zero298 Dec 16 '19 at 17:42
  • Is `server.listen()` aware of the cluster child status so it doesn't really do a from scratch listen, but just communicates with the parent? That's all I can think of here. – jfriend00 Dec 16 '19 at 17:49
  • Yeah, that's what seems to be going on. `server.listen()` has lots of [cluster-aware code](https://github.com/nodejs/node/blob/f5fe38400def6e0c73b27db436af5df914afe908/lib/net.js#L1388) and does something different when it's a cluster worker. Boy, that is a misleading way to implement things. – jfriend00 Dec 16 '19 at 17:51
  • Agreed @jfriend00. If the `listen` function is called from a child worker, I was confused about how the master process would be doing the actual "listening". Also, what would happen if each child process called `listen()` with a different port number? You would think instead you'd call `cluster.listen()` and pass in a list of child worker handles so it could distribute the calls. – Ryan Griggs Dec 16 '19 at 18:35
  • @RyanGriggs - yeah, `cluster.listen()` would have made it a lot simpler for us to understand, but there must be a reason they wanted to intercept all `.listen()` calls in the child. Yeah, I don't know how the master is doing the listening. I didn't continue to follow it all that far. – jfriend00 Dec 16 '19 at 18:36
  • Also, what about performance? Is there a performance hit because a single process is doing all the listening, then passing the connections to the workers? I guess that's the way it has to be for any server listening on a single port, right? so it's technically not multi-threaded until after the connection is passed to a worker? – Ryan Griggs Dec 16 '19 at 18:37
  • 1
    @RyanGriggs - Well, that's just the limitation of a modern OS. There really is only one true listener on a port. Fortunately, all the master is generally doing is dispatching which is already bottlenecked a bit in the TCP stack so it shouldn't be adding to the bottleneck in any harmful way. Remember, it's the child processes that are doing all the real work here. The master should have plenty of free time. – jfriend00 Dec 16 '19 at 18:40
  • 2
    @RyanGriggs - The hardware-assisted way to do this at high scale is to use a specialized load balancer or proxy like nginx to distribute the requests among a set of listening servers. Then, you can load balance not only among different processes on the same server, but among different servers to. So, there are levels of scale. Clustering on a single server is the next tier of scale up from a single node.js process. There are certainly higher tiers to go to by involving more hardware and eventually even involving edge servers and multiple data centers. – jfriend00 Dec 16 '19 at 18:41