Parallel Request at different paths in NodeJS: long running path 1 is blocking other paths

Question

I am trying out simple NodeJS app so that I could to understand the async nature.

But my problem is as soon as I hit "/home" from browser it waits for response and simultaneously when "/" is hit, it waits for the "/home" 's response first and then responds to "/" request.

My concern is that if one of the request needs heavy processing, in parallel we can't request another one? Is this correct?

    app.get("/", function(request, response) {
        console.log("/ invoked");
        response.writeHead(200, {'Content-Type' : 'text/plain'});
        response.write('Logged in! Welcome!');
        response.end();
    });

    app.get("/home", function(request, response) {
        console.log("/home invoked");
        var obj = {
            "fname" : "Dead",
            "lname" : "Pool"
        }
        for (var i = 0; i < 999999999; i++) {
            for (var i = 0; i < 2; i++) {
                // BS
            };  
        };
        response.writeHead(200, {'Content-Type' : 'application/json'});
        response.write(JSON.stringify(obj));
        response.end();
    });

Answer 1

Good question, Now, although Node.js has it's asynchronous nature, this piece of code:

for (var i = 0; i < 999999999; i++) {
    for (var i = 0; i < 2; i++) {
        // BS
    };  
};

Is not asynchronous actually blocking the node main thread. And therefore, all other requests has to wait until this big for loop will end.

In order to do some heavy calculations in parallel I recommend using setTimeout or setInterval to achieve your goal:

var i=0;
var interval = setInterval(function() {
   if(i++>=999999999){
       clearInterval(interval);
   }
   //do stuff here
},5);

For more information I recommend searching for "Node.js event loop"

Answer 2

As Stasel, stated, code running like will block the event loop. Basically whenever javascript is running on the server, nothing else is running. Asynchronous I/O events such as disk I/O might be processing in the background, but their handler/callback won't be call unless your synchronous code has finished running. Basically as soon as it's finished, node will check for pending events to be handled and call their handlers respectively.

You actually have couple of choices to fix this problem.

1. Break the work in pieces and let the pending events be executed in between. This is almost same as Stasel's recommendation, except 5ms between a single iteration is huge. For something like 999999999 items, that takes forever. Firstly I suggest batch process the loop for about sometime, then schedule next batch process with setimmediate. setimmediate basically will schedule it after the pending I/O events are handled, so if there is not new I/O event to be handled(like no new http requests) then it will executed immediately. It's fast enough. Now the question comes that how much processing should we do for each batch/iteration. I suggest first measure how much does it on average manually, and for schedule about 50ms of work. For example if you have realized 1000 items take 100ms. Then let it process 500 items, so it will be 50ms. You can break it down further, but the more broken down, the more time it takes in total. So be careful. Also since you are processing huge amount of items, try not to make too much garbage, so the garbage collector won't block it much. In this not-so-similar question, I've explained how to insert 10000 documents into MongoDB without blocking the event loop.

2. Use threads. There are actually a couple nice thread implementations that you won't shoot yourself in foot with them. This is really a good idea for this case, if you are looking for performance for huge processings, since it would be tricky as I said above to implement CPU bound task playing nice with other stuff happening in the same process, asynchronous events are perfect for data-bound task not CPU bound tasks. There's nodejs-threads-a-gogo module you can use. You can also use node-webworker-threads which is built on threads-a-gogo, but with webworker API. There's also nPool, which is a bit more nice looking but less popular. They all support thread pools and should be straight forward to implement a work queue.

3. Make several processes instead of threads. This might be slower than threads, but for huge stuff still way better than iterating in the main process. There's are different ways. Using processes will bring you a design that you can extend it to using multiple machines instead of just using multiple CPUs. You can either use a job-queue(basically pull the next from the queue whenever finished a task to process), a multi process map-reduce or AWS elastic map reduce, or using nodejs cluster module. Using cluster module you can listen to unix domain socket on each worker and for each job just make a request to that socket. Whenever the worker finished processing the job, it will just write back to that particular request. You can search about this stuff, there are many implementations and modules existing already. You can use 0MQ, rabbitMQ, node built-in ipc, unix domain sockets or a redis queue for multi process communications.