1

Is the CPU keeping on checking some status to see if the I/O call is returned?

I am asking this question just because I want to understand why async mode server (like nodejs) can handle more concurrent request.

If the CPU is just idle when a synchronized I/O call is not returned, then it would be fine that the server can start a new thread for handing new request as it still have enough CPU resources.

vancewang
  • 5,120
  • 3
  • 15
  • 14
  • 2
    Using an event-based model (hidden behind whatever abstraction, be it asynchronous events or continuations or..) has advantages over a thread-based model when there are *many* connections - primarily there is less overhead associated with each connection (in terms of memory usage and OS scheduling). This is talked about a bit [this old Jetty documentation](http://docs.codehaus.org/display/JETTY/Continuations). – user2864740 Sep 06 '14 at 10:28
  • thanks, it makes sense. so it is not the CPU resource makes it different when there are many connections in two mode. – vancewang Sep 06 '14 at 14:06

1 Answers1

2

If you do synchronous I/O, then your process (or thread) is blocked until the operation completes. That's why it's called synchronous. What is the CPU doing during that time? Servicing other processes. The same way most modern operating systems "schedule" work from multiple processes so none of them is completely deprived even if there are more jobs than processors, scheduling happens when I/O causes a process to no longer be runnable. If no runnable process is available, the processor can just idle for a while.

Asynchronous (aka non-blocking, evented) I/O is different. There, within a single process you have the ability to do more work while waiting for I/O events. It's like you're scheduling work within your own process, rather than depending on the OS to choose what work to do while you wait for I/O. Node.js can handle many concurrent connections because all I/O is done in non-blocking mode, so a single Node.js process can achieve a high level of utilization of a single processor core, and is never stuck waiting for anything (except truly CPU-bound computation, of course, but most Node.js programs don't have terribly much of that).

John Zwinck
  • 239,568
  • 38
  • 324
  • 436
  • Thanks. But as there is no extra CPU resouce is used when waiting the sync call's return, I understand it is possible to start a new thread (within that process) to handing the new requests, so even for sync mode sever, we still can handling as many requests as that for async mode server. Why do they say nodejs can handle more request concurrently? – vancewang Sep 06 '14 at 13:18
  • 5
    while waiting, synchronous I/O allows you to reuse the CPU, asynchronous I/O allows you to reuse the THREAD. Creating new thread for each request has OS overhead, having too many threads has an overhead too. Doing multiple async I/O on the same thread doesn't. That's the gain you get. – Sedat Kapanoglu Sep 06 '14 at 14:36
  • @caesarwang: in theory both the async model of Node.js and the thread-per-client model can handle many requests concurrently. In practice, as ssg points out, threads have more overhead, so do not scale as well as I/O multiplexing via epoll(), kqueue(), WaitForMultipleEvents(), etc. Of course, to know which system can truly handle the most requests, one would have to test them. A multi-core system with a single server having to do a lot of CPU-intensive calculations for each client might actually be better done with threads...but that is not the common use case. – John Zwinck Sep 06 '14 at 14:42
  • @JohnZwinck while in theory async model of Node.js should provide better throughput without the need for `nginx`-like load balancer, this programming style generates infamous [callback hell](http://stackoverflow.com/questions/18095107/callback-hell-in-nodejs) which in turn complicates the software and makes the maintenance expensive and that is why "node.js threads please" questions appear from time to time also on SO, like this one: http://stackoverflow.com/questions/23613357/a-version-of-node-js-that-is-multi-threaded – xmojmr Sep 07 '14 at 06:34