How to figure out order of execution for Node.js?

Question

I am currently trying to test some Node.js code. Right now I am trying to figure out why my code console.log()'s in different order relative to the code execution order. This is regarding OS related Node.js function calls.

My script does this in order:

//First
fs.readdir(){ //read from a directory all the available files
   console.log some stuff #a
}

//Second
fs.readFile(){ //read entire text file
   console.log some stuff #b
}

//Third
//create readline interface using fs.createReadStream(filedir) then

interface{ //read from text file line by line
   console.log some stuff #c
}

Function a uses the path: __dirname + '/text_file/'
Function b and c uses the path: __dirname, '/text_file/text1.txt'

Output:

a
c
b

Can someone explain why this order happens? I am not an expert in Operating System or what is happening in the background with Node.js. Will this ordering mess up my app if I were to rely on this?

Answer 1

If you start multiple asynchronous operations like this one after the other, the completion order is indeterminate. These operations run independently of your Javascript. They are what is called "non-blocking" which means they return immediately from the function call and your Javascript keeps executing. Some time later, they finish, insert an event in the Javascript event queue and then the callback you passed to them gets called. The "asynchronous, non-blocking" aspect of these functions means that they don't "wait" for the operation to complete before returning.

The order of completion of multiple asynchronous operations that are all in flight at the same time will just depend upon which one finishes it's work before the others.

If you want them to run in sequence, then you have to specifically code them to run sequentially by starting the second operation in the completion callback of the first and starting the third one in the completion callback of the third.

For example:

fs.readdir(somePath, function(err, files) {
    if (err) {
        console.log(err);
    } else {
        fs.readFile(someFilename, function(err, data) {
            if (err) {
                console.log(err);
            } else {
                // done with both asynchronous operations here
                console.log(data);
            }
        });
    }
});

Some other relevant references on the topic:

How to let asynchronous readFile method follow order in node.js

How to synchronize a sequence of promises?

How do I set a specific order of execution when asynchronous calls are involved?

How does Asynchronous Javascript Execution happen? and when not to use return statement?

Single thread synchronous and asynchronous confusion

Why do callbacks functions allow us to do things asynchronously in Javascript?

Answer 2

readdir and readFile are asynchronous, that is, they do processing outside the main flow of code. That is why you get to give them a callback, which is executed once the operation finishes, which might take a long time (in a computer's notion of time).

You cannot rely on asynchronous functions being completed before the next synchronous line of code executes at all. That means, your code will eventually end up looking like this:

fs.readdir(someDir, function(err, files) {
    if (err) return;
    // do stuff with the list of files in the directory

    fs.readFile(someFile, function(err, contents) {
        if (err) return;
        // do stuff with the file's contents
    });
});

Why asynchronous, you might ask? Well, maybe you can go about doing other things while the (very slow) disk spins around trying to find your data. Stopping the whole program while the disk reads would be a very notable waste of time, so Node makes such operations that depend on disk or network asynchronous, which frees you to continue with any other processing until they are done.

Of course, that means you have to write code that depends on the output of such asynchronous operations inside the callback.

---

In another note:

Hopefully you can see that for big applications, this style of coding is very cumbersome, since the main code would ever go more to the right, and error tracking is tricky. That is why Promises were invented, and I would recommend you learn them, but after you understand normal asynchronicity.

Answer 3

Unlike traditional programming languages, JavaScript (and its variants like NodeJS) run the code in parallel, each asynchronous function runs independently from the others and because of this the program wont run in sequence, this makes the execution faster but more difficult to control

to handle synchronous functions there are libraries like ASYNC that will allow you to control what to do when one or more functions end, and will help you limit or eliminate the callback hell

https://www.sohamkamani.com/blog/2016/03/14/wrapping-your-head-around-async-programming/

Answer 4

All these functions are asynchronous and will be performed in parallel. fs.readFile will callback will execute after the whole file is read as for fs.createReadStream callback will execute when the first chunk of the stream is readable, and that´s faster than the first one.