0

I have a socket.io server on my node.js API.

When the node.js server is doing nothing, the latency of pong event is usually 5-10 milliseconds. However, when the node.js server is processing some very complex http request, the latency jumps to more than 100 or even over 5000 causing ping timeout.

I am curious about what is effecting the latency?

And, our framework requires a stable socket connection to send infos of a long http request to same socket as the request was made. How can we make connections stable without very long pingTimeout and pingInterval.

zhangjinzhou
  • 2,461
  • 5
  • 21
  • 46

1 Answers1

2

It really depends what "processing some very complex http request" is doing. node.js runs your Javascript as a single thread. That means it can literally only do one thing at a time. But, since many things that servers do are I/O related (read from a database, get data from a file, get data from another server, etc...) and node.js uses event-driven asynchronous I/O, it can often have many balls in the air at the same time so it appears to be working on lots of requests at once.

But, if your complex http request is CPU-intensive, using lots of CPU, then it's hogging the single Javascript thread and nothing else can get done while it is hogging the CPU. That means that all incoming HTTP or socket.io requests have to wait in a queue until the one node.js Javascript thread is free so it can grab the next event from the event queue and start to process that incoming request.

We could only really help you more specifically if we could see the code for this "very complex http request".

The usual way around CPU-hogging things in node.js is to offload CPU-intensive stuff to other processes. If it's mostly just this one piece of code that causes the problem, you can spin up several child processes (perhaps as many as the number of CPUs you have in your server) and then feed them the CPU-intensive work and leave your main node.js process free to handle incoming (non-CPU-intensive) requests with very low latency.

If you have multiple operations that might hog the CPU, then you either have to farm them all out to child processes (probably via some sort of work queue) or you can deploy clustering. The challenge with clustering is that a given socket.io connection will be to one particular server in your cluster and if it's that process that just happens to be executing a CPU-hogging operation, then all the socket.io connections assigned to that server would have bad latency. So, regular clustering is probably not so good for this type of issue. The work-queue and multiple specialized child processes to handle CPU-intensive work are probably better because those processes won't have any outside socket.io connections that they are responsible for.

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • This totally makes sense. My service sends a lot of database requests or http requests at a time. Once all the promises are resolved, the process will go to next step. I will make the parellel processes synchronously and observe the latency. Hopefully we can decide which part of the service cause the problem. – zhangjinzhou Jul 25 '18 at 01:13
  • Can streaming be CUP-intensive? Another of my services processes gigbytes of data and output millions of records to a csv file. – zhangjinzhou Jul 25 '18 at 16:45
  • @zhangjinzhou - Could be. It depends on how much processing is done to the data. Processing gigabytes of data and outputting millions of records to a csv file sounds like it could use a bunch of CPU just dealing with all the data conversion. You'd have to just watch the CPU utilization while running it to see. It's also doing a lot of I/O which uses some CPU, but node.js is generally very efficient with I/O. It is critical that you are using only asynchronous file I/O though. – jfriend00 Jul 25 '18 at 17:05
  • They are spatial files. I have to fist take the streams and output them to disk. Then open the files, read geometries and attributes, transfer them to text and output to text. I did a test just now. They are really cpu consuming. My computer have 4 cpus. Usually there is only one cpu working at 100% and others at about 10%. When doing the process, all the three jumped to about 50%-70%. Not to mention this is just a 300M file. I am actually using sync version of `file` to do I/O. Why do synchronous file I/O raise CPU consuming? – zhangjinzhou Jul 25 '18 at 17:23
  • @zhangjinzhou - Synchronous file I/O blocks the ONE and only one node.js thread so it can do nothing else even when just waiting for the disk. It doesn't necessarily use more CPU, but it blocks node.js from doing anything else during disk I/O. It appears your streaming does use a bunch of CPU. – jfriend00 Jul 25 '18 at 17:35
  • I really appreciate your help! I have another question about this problem. It has not been answered even with bounty. Could you kindly post something there so I can give it credit? I have been doing node.js for 1 year and got a lot of helps from you sir. You are so knowledgeable and patient! – zhangjinzhou Jul 25 '18 at 17:37
  • @zhangjinzhou - Sure. I posted an answer to your other question. – jfriend00 Jul 25 '18 at 17:41
  • 1
    @zhangjinzhou - OK, then synchronous file I/O completely explains why your server is non-responsive. It's blocking the only thread node.js has to run your Javascript so your server can do nothing else during your file operations. You need to convert to asynchronous file I/O. You may find streams are easiest to use. – jfriend00 Jul 25 '18 at 17:44
  • Exactly. I realized this just now. Does sync/await also block the thread? – zhangjinzhou Jul 25 '18 at 17:47
  • @zhangjinzhou - `Async/await` does not block the thread. It blocks the remaining execution of that function, but the main JS thread is able to handle other events and run other code while `await` waits. You will have to use I/O functions that return promises in order to use them with `await`. If you get stuck converting your I/O code to async, you can post a new question about that. – jfriend00 Jul 25 '18 at 18:11