31

The following example is given in a Node.js book:

var open = false;

setTimeout(function() {
  open = true
}, 1000)

while (!open) {
  console.log('wait');
}

console.log('open sesame');

Explaining why the while loop blocks execution, the author says:

Node will never execute the timeout callback because the event loop is stuck on this while loop started on line 7, never giving it a chance to process the timeout event!

However, the author doesn't explain why this happens in the context of the event loop or what is really going on under the hood.

Can someone elaborate on this? Why does node get stuck? And how would one change the above code, whilst retaining the while control structure so that the event loop is not blocked and the code will behave as one might reasonably expect; wait will be logged for only 1 second before the setTimeout fires and the process then exits after logging 'open sesame'.

Generic explanations such as the answers to this question about IO and event loops and callbacks do not really help me rationalise this. I'm hoping an answer which directly references the above code will help.

JJJ
  • 32,902
  • 20
  • 89
  • 102
codecowboy
  • 9,835
  • 18
  • 79
  • 134
  • 1
    Possible duplicate of [How the single threaded non blocking IO model works in Node.js](http://stackoverflow.com/questions/14795145/how-the-single-threaded-non-blocking-io-model-works-in-node-js) – thefourtheye Jan 16 '16 at 07:08
  • 2
    I don't agree that this is a duplicate. It references a specific piece of code and a specific explanation of this piece of code. It also asks how the code could be rewritten. – codecowboy Jan 16 '16 at 07:17

6 Answers6

50

It's fairly simple really. Internally, node.js consists of this type of loop:

  • Get something from the event queue
  • Run whatever task is indicated and run it until it returns
  • When the above task is done, get the next item from the event queue
  • Run whatever task is indicated and run it until it returns
  • Rinse, lather, repeat - over and over

If at some point, there is nothing in the event queue, then go to sleep until something is placed in the event queue or until it's time for a timer to fire.


So, if a piece of Javascript is sitting in a while() loop, then that task is not finishing and per the above sequence, nothing new will be picked out of the event queue until that prior task is completely done. So, a very long or forever running while() loop just gums up the works. Because Javascript only runs one task at a time (single threaded for JS execution), if that one task is spinning in a while loop, then nothing else can ever execute.

Here's a simple example that might help explain it:

 var done = false;

 // set a timer for 1 second from now to set done to true
 setTimeout(function() {
      done = true;
 }, 1000);

 // spin wait for the done value to change
 while (!done) { /* do nothing */}

 console.log("finally, the done value changed!");

Some might logically think that the while loop will spin until the timer fires and then the timer will change the value of done to true and then the while loop will finish and the console.log() at the end will execute. That is NOT what will happen. This will actually be an infinite loop and the console.log() statement will never be executed.

The issue is that once you go into the spin wait in the while() loop, NO other Javascript can execute. So, the timer that wants to change the value of the done variable cannot execute. Thus, the while loop condition can never change and thus it is an infinite loop.

Here's what happens internally inside the JS engine:

  1. done variable initialized to false
  2. setTimeout() schedules a timer event for 1 second from now
  3. The while loop starts spinning
  4. 1 second into the while loop spinning, the timer is ready to fire, but it won't be able to actually do anything until the interpreter gets back to the event loop
  5. The while loop keeps spinning because the done variable never changes. Because it continues to spin, the JS engine never finishes this thread of execution and never gets to pull the next item from the event queue or run the pending timer.

node.js is an event driven environment. To solve this problem in a real world application, the done flag would get changed on some future event. So, rather than a spinning while loop, you would register an event handler for some relevant event in the future and do your work there. In the absolute worst case, you could set a recurring timer and "poll" to check the flag ever so often, but in nearly every single case, you can register an event handler for the actual event that will cause the done flag to change and do your work in that. Properly designed code that knows other code wants to know when something has changed may even offer its own event listener and its own notification events that one can register an interest in or even just a simple callback.

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • thanks @jfriend00. Thats a great answer and just what I was looking for. I hope others will upvote it too, not that you need the rep ;) – codecowboy Jan 16 '16 at 12:03
  • So I get a very long run around the bush that while loops block in js. – marshal craft Sep 08 '17 at 06:27
  • @marshalcraft - ANY loop in Javascript blocks. Javascript is single threaded and event driven so as long as a loop is running, nothing else except the code in the loop can run. And, it's not just loops. Any long running function keeps other things from running. Single threaded - Javascript can only do one synchronous thing at a time. All loops are synchronous. In case you missed it, the summary is in my third paragraph after it explains how the event queue works (which is the key to understanding all this). I don't think that's a very long run around the bush. – jfriend00 Sep 08 '17 at 06:41
  • I apologize jftiend. I was over critical. So non asynchronous functions block. But really a while loop in a asynchronous function would still block. Seems a while loop from start to exit is a single unit of asynchronous execution. I understand js is single threaded. But it doesn't necessarily execute code inline, so it pauses in some cases to continue asynchronous code, or events. Being as it doesn't use time to dictate how much to execute, it must use "atomic" or basic units of execution. Like run global scope code for a little, pause to handle event or do some work on asynchronous function. – marshal craft Sep 08 '17 at 11:03
  • It is conceivable that the js interpreter or runtime could do this on one thread. Also forgive my probably poor js verbage and terminology. I'm not strong with js. – marshal craft Sep 08 '17 at 11:05
  • Is it possible you can point out a good description of what units are used to switch between sections of js code. Like for example js internal syntatic blocks (basic operations), while{}, if{}, _*_, (), etc. could be basic operations which block or always must complete. I tried to find such articles in emca 262 but it's intense naming scheme makes it difficult to find specific parts pertaining to this topic with out having to read several hundred pages. – marshal craft Sep 08 '17 at 11:14
  • @marshalcraft - There are no units. There is no time slicing. A given piece of Javascript runs until it returns control back to the system, where the system then pulls the next event from the event queue. The entire thing is event driven, one event at a time. Asynchronous operations such as `setTimeout()` start themselves and then return. Some time later, the native code behind the `setTimeout()` inserts an event into the event queue. When the JS interpreter is done with previous Javascript, it will pull that event out of the event queue and call the callback associated with it. – jfriend00 Sep 08 '17 at 13:09
  • @marshalcraft - I can't explain all this in a few comments. Read these: [Javascript event queue and setTimeout](https://stackoverflow.com/questions/34691428/javascript-event-queue-and-settimeout/34691484#34691484), [How does Javascript handle Ajax in the background](https://stackoverflow.com/questions/7575589/how-does-javascript-handle-ajax-responses-in-the-background/7575649#7575649) and [Can JS event handlers interrupt one another](https://stackoverflow.com/questions/8016001/can-js-event-handlers-interrupt-execution-of-another-handler/8016185#8016185) and ask a new question if still confused – jfriend00 Sep 08 '17 at 13:19
  • Ok after I read a lot i get it. JavaScript isn't asynchronous. It run synchronously to end of source (global) then starts checking events and handling them. Thanks and sorry for my missunderstanding, nobody said like this and I had a whole other rationing of it. – marshal craft Sep 08 '17 at 16:36
  • @marshalcraft - Yep, that's pretty much it. – jfriend00 Sep 08 '17 at 16:44
  • @jfriend00 `1 second into the while loop spinning, the timer fires internally to the JS engine and the timer callback is added to the event queue. `-->I don't think js engine had an chance to add **the timer callback** to **the event queue**. If js engine had an chance to do that, it could execute callback that assign done to true too; The right should be that **the entire js event loop block at the while loop entriely**, js engine had no chance to do anything else, because it it single threaded. – waterd Nov 16 '18 at 08:56
  • @jfriend00 Or `timer callback` is added to a list when `setTimeout` executed, but it had no chance to know whether it should execute the `timer callback`. – waterd Nov 16 '18 at 09:08
  • @waterd - It isn't the JS engine that adds the timer callback to the event queue. It would be a native code sub-system that manages timers. While Javascript is only run in a single thread, there are other actual threads that do other things (like manage various asynchronous operations). Those threads don't directly run Javascript. They insert things into the event queue that the single thread of Javascript can pull out of the event queue when it's done doing things before it. – jfriend00 Nov 16 '18 at 10:07
  • @jfriend00 Thanks. I thought js event loop is something like libevent event loop. Maybe I should do more research. But I doubt that why there are other threads in js engine. – waterd Nov 16 '18 at 13:28
  • @waterd - The actual event queue/loop is more complicated than portrayed here as it has many different queues and when the JS engine is done running a piece of Javascript, it returns control back to the event loop and it goes to whatever the next stage of the event loop is and looks for events there. If none are found, it advances to the next stage of the event loop. One of those stages is a check for any timers that are ready to run and there is a set of native code that manages all running timers and determines when one of the timers should fire and trigger a call from the event loop. – jfriend00 Nov 16 '18 at 16:54
  • @jfriend00 Thanks for giving more details. I am newbie actually. This will encourage me to learn more about *js engine implementation*. By the way, I am struggling on [how-to-wait-on-asynchronous-javascript-function-without-async-await-support](https://stackoverflow.com/questions/53368026/how-to-wait-on-asynchronous-javascript-function-without-async-await-support). – waterd Nov 19 '18 at 05:40
  • @waterd - I added some comments to your other question. – jfriend00 Nov 19 '18 at 05:58
  • It does not say how change to code to make it works – Perry Aug 02 '19 at 08:06
  • @Perry - In general, you don't use a `while` loop like this in Javascript to spin waiting for something. So, you don't do it this way at all. Javascript is an event driven language so you register an event handler for whatever events will occur in the future that change the state of things and you do something on those events rather than spinning checking state. This is make up code that doesn't have any attachment to a real world situation. If it was a real world situation (where there were events involved), we could advise how to best do this using those events. – jfriend00 Aug 02 '19 at 09:35
  • @Perry - Said differently, in the real world, some event would cause the `done` flag to change. You'd register an event handler for that event and do your work there or examine the `done` flag there. – jfriend00 Aug 02 '19 at 09:35
  • Real world example, service that is preparing the output, because is slow the client decide to stop waiting and get the current output... it is not a while waiting for an event, it is a while inside a function that can be stopped by an external event... look https://stackoverflow.com/questions/56061297/node-js-allow-process-update-during-loop – Perry Aug 09 '19 at 07:43
9

This is a great question but I found a fix!

var sleep = require('system-sleep')
var done = false

setTimeout(function() {
  done = true
}, 1000)

while (!done) {
  sleep(100)
  console.log('sleeping')
}

console.log('finally, the done value changed!')

I think it works because system-sleep is not a spin wait.

danday74
  • 52,471
  • 49
  • 232
  • 283
  • 3
    I wonder no one upvoted the only solution on this page which simply works, apart from all the PHD talks. – jolly Oct 19 '17 at 06:12
  • 2
    I later found that this is the only solution that works BUT it only works because it is a C++ hack, this should not be possible in Node.js - this will work on the majority of platforms, however, it will fail on some platforms so should be used with caution - great for dev use however – danday74 Oct 19 '17 at 08:48
3

There is another solution. You can get access to event loop almost every cycle.

let done = false;

setTimeout(() => {
  done = true
}, 5);

const eventLoopQueue = () => {
  return new Promise(resolve => 
    setImmediate(() => {
      console.log('event loop');
      resolve();
    })
  );
}

const run = async () => {
  while (!done) {
    console.log('loop');
    await eventLoopQueue();
  }
}

run().then(() => console.log('Done'));
AlbertS
  • 706
  • 5
  • 12
2

Node is a single serial task. There is no parallelism, and its concurrency is IO bound. Think of it like this: Everything is running on a single thread, when you make an IO call that is blocking/synchronous your process halts until the data is returned; however say we have a single thread that instead of waiting on IO(reading disk, grabbing a url, etc) your task continues on to the next task, and after that task is complete it checks that IO. This is basically what node does, its an "event-loop" its polling IO for completion(or progress) on a loop. So when a task does not complete(your loop) the event loop does not progress. To put it simply.

codecowboy
  • 9,835
  • 18
  • 79
  • 134
tsturzl
  • 3,089
  • 2
  • 22
  • 35
  • 2
    @codecowboy In Node.js there is only one main thread. So all the code has to be executed by that thread only. The `setTimeout` will schedule the function to be executed after 1 second. So it will not execute the function immediately and move to the next executable statement. It finds a `while` loop. Since it is single threaded, only when the current task is completed, Node.js can pick the next item from the queue. But then the `while` runs infinitely and so the function registered by `setTimeout` never gets a chance to execute. – thefourtheye Jan 16 '16 at 07:41
0

because timer needs to comeback and is waiting loop to finish to add to the queue, so although the timeout is in a separate thread, and may indeed finsihed the timer, but the "task" to set done = true is waiting on that infinite loop to finish

Ziyi Wang
  • 75
  • 1
  • 5
0
var open = false;
const EventEmitter = require("events");
const eventEmitter = new EventEmitter();

setTimeout(function () {
  open = true;
  eventEmitter.emit("open_var_changed");
}, 1000);

let wait_interval = setInterval(() => {
console.log("waiting");
}, 100);

eventEmitter.on("open_var_changed", () => {
clearInterval(wait_interval);
console.log("open var changed to  ", open);
});

this exemple works and you can do setInterval and check if the open value changed inside it and it will work

Salah Ben Bouzid
  • 381
  • 4
  • 10