10

I have started reading a lot about Node JS lately and one thing I am not able to clearly understand from differentiation perspective is what is the real difference between how I/O is handled by Asynchronous Vs Synchronous call.

As I understand, in a multi threaded synchronous environment , if I/O is started , the running thread is preempted and moves back to a waiting state.So essentially this is same as what happens with NodeJS asynchronous I/O call. In Node JS, when I/O is called, the I/O operation is moved out of current running thread and sent to event De-multiplexer for completion and notification. As soon as I/O is complete the callback method is pushed to event Queue for further processing.

So , the only difference I see is that in Node JS we are saving memory (due to multiple call stacks owned by each thread) and CPU ( saved because of no context switching). If I just consider that I have enough memory to buy , does saving of CPU due to context switching alone is making the huge performance difference?

If my above understanding is not correct, how does I/O handling is any different between a java thread Vs Node JS w.r.t to keeping the CPU busy and not wasting CPU cycles.Are we saving only context switching CPU cycles with Node JS or there is more to that?

Based on the responses , I would like to add another scenario:

Request A , Request B comes to J2ee server at the same time. Each request takes 10 ms to complete in this multi threaded environment.Out of 10 ms , 5 ms is spent in executing the code logic to compute some logic and 5 ms is spent in I/O for pulling a large dataset from a DBMS.The call to DBMS is the last line of the code after which the response should be sent to the client.

If same application converted to a node JS application, this is what might happen

  1. Request A comes, 5 ms is used for processing the request.

  2. DBMS call is hit from the code but it's non blocking.So a callback method is pushed to event Queue.

  3. After 5 ms, Request B is served and again request B is pushed to event Queue for I/O completion.Request B takes 5 ms for processing.
  4. Event Loop runs, pickups callback handler for request A, which then sends the response to client.So the response is sent after 10 ms because Req A and Req B both took 5 ms for synchronous code block processing.

Now where is the time saved in such a scenario?Apart from context switching and creating 2 threads. Req A & Req B both took 10 ms anyway with Node JS. ?

learnerFromHell
  • 183
  • 1
  • 9
  • 1
    Your example is a single series of events that depends on the completion of the last before the next. Node is not going to speed that up, in fact its theoretically impossible to do that because you require the last value for the next task. What node is good at is say you wanted to handle many of these events you can do that concurrently, rather than only being able to handle one at a time. Your example doesn't make much sense in an aspect of concurrency or parallelism. – tsturzl Mar 16 '16 at 06:05
  • No threads were ever created, no contexts were switched. I think this answer should be closed because I don't think it can be answered. – tsturzl Mar 16 '16 at 06:06
  • Time is saved because instead of blocking execution it can move on to handle other things, but you can't move on in the same serial task without the value from the previous task. Its a better approach to threading as I explained in my comments on the answer below. Context switching has memory and CPU overhead. It's heavy and not well suited for concurrent IO. An eventful or streaming style of concurrency like nodejs provides is lightweight and perfectly suited to IO, same goes for something like coroutines. These require less allocations, context switching, and memory. – tsturzl Mar 16 '16 at 06:10
  • Essentially sacrificing low level control for performance and reduced overhead. Context switching is heavy, allocating that memory is very heavy(usually why you have a threadpool), and for IO those low level controls are not needed. Threads are essentially schedule similarly to processes, they switch context and that requires many CPU cycles on its own. Threads are expensive, even in a thread pool. You save more than memory, you save a lot of useless work switching contexts which is no light task. – tsturzl Mar 16 '16 at 06:18
  • http://learn-gevent-socketio.readthedocs.org/en/latest/general_concepts.html – tsturzl Mar 16 '16 at 06:20

1 Answers1

5

As I understand, in a multi threaded synchronous environment , if I/O is started , the running thread is preempted and moves back to a waiting state.So essentially this is same as what happens with NodeJS asynchronous I/O call.

Nope, in NodeJS an asynchronous I/O call is a non-blocking I/O. Which means that once the thread has made an I/O call it doesn't wait for the I/O to complete and move on to the next statement/task to execute.

Once the I/O completes it picks up the next task from event-loop-queue and eventually executes callback handler which was given to it while making the I/O call.

If I just consider that I have enough memory to buy , does saving of CPU due to context switching alone is making the huge performance difference?

Apart from this, saving is also coming from these two things

  • Not-Having-To-Wait for the I/O to complete
  • Not-Having-To-Make-Threads since threads are limited, so the system's capacity is not limited by how many threads it can make.

Apart from context switching and creating 2 threads. Req A & Req B both took 10 ms anyway with Node JS. ?

You are discounting one thing here - Thread is getting two request one after the other after a specific interval. So if one thread is going to take 10 seconds, then it will take a new thread to execute the second request. Extapolate this to thousands of requests and your OS having to make thousands of threads to deal with so many concurrent users. Refer to this analogy.

Community
  • 1
  • 1
gurvinder372
  • 66,980
  • 10
  • 72
  • 94
  • But the point was the CPU is not ideal anyway in a multi threaded environment, it picks up another ready to run thread just like in Node JS? So for running the code , the CPU is getting utilized optimally. Is there any difference between the two? – learnerFromHell Mar 16 '16 at 05:29
  • @learnerFromHell Difference comes due - 1) Less context switching 2) Less threads to make 3) Less waiting for CPU cycles to be utilized since the burden of disk operations (which are less CPU intensive but may have high time cost anyways) is not born by the nodejs app. – gurvinder372 Mar 16 '16 at 05:35
  • @learnerFromHell IO usually has very little to do with CPU. So maximizing CPU potential with multi-threading in this scenario is completely useless. Node does not context switch or have more than a single thread. There are essentially 2 ways to wait for IO. Blocking, which means that the thread of execution gets halted till a value is returned. Or you can not block and continuously check a buffer to see if a response has come, and even process it fragment by fragment as data comes in, which is what node essentially does. – tsturzl Mar 16 '16 at 05:40
  • ok, so I already mentioned in my question that point 1 & 2 are accepted benefits of Node JS . Point 3 I didn't understand well enough because CPU is anyway not utilized for actual I/O . I think the way I understand this is that there is another delay factor which can come in to picture is when is the preempted thread getting a chance to execute again. Is that a big enough factor? The case will be same for event waiting in event Queue. – learnerFromHell Mar 16 '16 at 05:42
  • tsturzl : If a request A is taking 10 ms for processing in multithreaded env, and it is waiting for 5 ms for IO to complete (thread is stuck, response is not sent), in Node JS, this 5 ms is saved? The response for the request won't be sent back until 5 ms is elapsed for the IO operation to complete anyway , because the response will be sent only when the callback for the IO is complete. Where am i going wrong? – learnerFromHell Mar 16 '16 at 05:46
  • They are completely different paradigms – tsturzl Mar 16 '16 at 05:47
  • 2
    The callback isn't the only logic. You're callback is only called upon completion, but say you have another request coming in, you can handle that on the same thread because your thread isn't blocked. – tsturzl Mar 16 '16 at 05:47
  • Lets say you want to open 3 files, and each file you want to HTTP post. You say read file 1 then issue a callback, move on to read file 2 issue another callback, and same for file 3. File 2, lets say, comes back first, but the others are still loading. You can then issue a POST request to send file 2. Then file 1 comes in, and you issue a POST. Then the response for file 2's POST comes back and you check for success. And do this until everything is done. You aren't stuck waiting on file 1 before issuing a read of file 2 or 3, and you can also send a POST request while waiting on those files. – tsturzl Mar 16 '16 at 05:52
  • This is all without threads. And this is actually done by the underlying OS. Its easiest to explain on a UNIX or Linux system where sockets and files are treated the same, because regardless the socket and file can both be non-blocking. Non-blocking means that instead of having a return value for that call, you are pointed to a buffer. This not only means you can check that buffer for completion and progress, but you can also treat it as a "stream" and process bits of data as it loads. OS handles this not by using threads but using its own scheduler. – tsturzl Mar 16 '16 at 05:54
  • Multi-thread is likely taking longer due to the overhead of switching contexts. Multi-threading is probably the worst approach for IO. Co-routines are another good paradigm to handle concurrent IO. They work in a similar manner, but offer more control and have a similar "flavor" as threads. – tsturzl Mar 16 '16 at 05:57
  • It is possible to multithread/process node, and its been done. It gives node parallelization abilities, but usually this just means having multiple instances of node running in different processes and just load balancing between them by evenly distributing the workload. Usually there is some kind of CPU logic(comparisons, parsing, arithmetics) that you perform on the IO results, and nodejs does no address this kind of computational concurrency, its model is directed at handling IO. And running multiple processes that are load balanced helps mitigate the delay from these tasks. – tsturzl Mar 16 '16 at 06:00
  • Also consider this. Your IO isn't going to be real time. In other words it might load, but checking it for completion might be further on in the event loop. So that IO wait could actually be longer, but it doesn't block your thread. – tsturzl Mar 16 '16 at 06:02
  • @learnerFromHell I/O tasks may not be CPU intensive but they take time. **Waiting time for the I/O to complete** is what you are saving. It can be a big factor (varies from context to context) if I/O operations like database call is time consuming. – gurvinder372 Mar 16 '16 at 07:20
  • @gurvinder372 Both Request A & B has hit the node server at the same instant. My example says that A has been picked up for servicing , so B needs to wait for 5 ms anyway.In Node JS, I cannot return the response unless my I/O is complete.So the I/O callbacks will be processed only after 10 ms. – learnerFromHell Mar 16 '16 at 17:50
  • 1
    @learnerFromHell If your app is going to spend 5ms in processing a every request before invoking an I/O, you get the capability of 200 concurrent requests per second with a single thread (which otherwise would take 200 different threads to achieve). However, if the I/O are time consuming the final throughput will obviously reduce. You obviously need to weigh in the pros and cons of this approach and design accordingly. Nodejs doesn't guarantee better performance for every scenario, it is designed to increase capacity and not necessarily performance. – gurvinder372 Mar 17 '16 at 05:34