How to let asynchronous readFile method follow order in node.js

Question

I have a code like this

var fs = require('fs');
console.log("1");
fs.readFile("./index.js", function(err, data) {
    if (err) {
         console.log("err");
    }
    console.log("2");
});
console.log("3");

and the result will be

1
3
2

But I want 1 2 3 I have look through filesystem's document and it says With the asynchronous methods there is no guaranteed ordering. so the document says if I want my code result 1 2 3 it should be

var fs = require('fs');
console.log("1");
fs.readFile("./index.js", function(err, data) {
    if (err) {
         console.log("err");
    }
    console.log("2");
    console.log("3");
});

But it's for short code if I'm building a system which has a lot of other function needs to be after this code we can't nested all in because the function need to be work after it will be use by other functions so is there a way to let the fs.readFile work in order but not incuding all the code need to work after it in fs code?

code I have issue

function servedatastart() {

    if (urlcontentbasic(urlslug) != "err") {
        responsearray["data"] = urlcontentbasic(urlslug);
    } else {
        // callback function
        console.log("urlex");
        urlex("none", urlslug); // this function I use fs.readFile(urlslug, function...
    }
    console.log("other function");
    //other function
    cuswriteheader();
    includescript();
    writefooter()
    res.end();   
}

Can you use ES7 await/async in this project? – Will Jul 05 '17 at 14:31 — Will, Jul 05 '17 at 14:31

Lena Kaplan · Accepted Answer · 2017-07-05T15:21:29.700

4

Use async.waterfall, it allows you to run asynchronous functions in order, each function depends on the previous one to complete. when the last function completes the main callback is called, each function can pass the result to the next one. check https://caolan.github.io/async/docs.html

var fs = require('fs');
var async = require('async');

async.waterfall([
    function (callback) {
        console.log("1");
        callback();
    },
    function (arg1, callback) {
        fs.readFile("./index.js", callback);
        console.log("2");
    },
    function (arg2, callback) {
        console.log("3");
        callback()
    }
], function(err, res){

});

edited Jul 05 '17 at 15:21

answered Jul 05 '17 at 14:43

Lena Kaplan

756
4
13

Does this work if I'm using it in express app will the variable `res` mess up with express app? – Andrew Young Jul 05 '17 at 14:47
shouldn't be a problem, I use it with express too, if you use it inside the route you can change the name to res1, or if you don't need it you can remove the res paramter from the main calback – Lena Kaplan Jul 05 '17 at 14:54
I match it with my real code like https://files.andrewyg.net/code/705-1.js but it show error of callback is not a function why? – Andrew Young Jul 05 '17 at 15:08
1

It depends on how your code, in helloheloo, handles the callback. If you return some parameters on the callback, you should add parameters in the signature of the next function, before the callback, check the good example they put in the documentation, it should help you understand how to use it. also please notice the edit in the code..the parameters arg1, arg2 before callbacks added – Lena Kaplan Jul 05 '17 at 15:23
I get the data from the `fs` following the instruction but `console.log` log out `` will browser decode it and become text? – Andrew Young Jul 05 '17 at 15:39
1

I don't think so. You need to specify the encoding in readFile. This is from fs documentation : If no encoding is specified, then the raw buffer is returned. If options is a string, then it specifies the encoding. Example: fs.readFile('/etc/passwd', 'utf8', callback); – Lena Kaplan Jul 05 '17 at 15:47

jfriend00 · Answer 2 · 2017-07-05T22:35:54.907

The simple solution is to just put the rest of your code that you want to execute after the fs.readFile() operation into a function and then insert that function into the async callback:

var fs = require('fs');
console.log("1");
fs.readFile("./index.js", function(err, data) {
    if (err) {
         console.log("err");
    }
    console.log("2");
    moreCode();
});

function moreCode() {
    console.log("3");
}

A bit more modern design is to use promises for all your asynchronous operations and then you can sequence things a bit easier.

const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));

console.log("1");
fs.readFileAsync("./index.js").then((data) => {
    console.log("2");
    return moreCode();
}).catch((err) => {
    // handle error here
});


function moreCode() {
    console.log("3");
}

Where the benefit of using promises really starts to take advantage is when you have multiple asynchronous operations that you want to sequence and each one returns a promise. Then, you can do that like this:

const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));

fs.readFileAsync("./index.js")
  .then(moreCode1)
  .then(moreCode2)
  .catch((err) => {
      // handle error here
  });

Using the code you've show for the urlex() function, here's how you can make all that work using promises:

// include infrastructure to automatically promisify the fs module
const Promise = require('bluebird');
const fs = Promise.promisifyAll(require('fs'));

function servedatastart() {

    Promise.resolve().then(function() {
        if (urlcontentbasic(urlslug) != "err") {
            // I'm assuming that urlcontentbasic is also async and returns a promise
            // like urlexbasic does
            return urlcontentbasic(urlslug);
        } else {
            // callback function
            console.log("urlex");
            return urlex("none", urlslug); 
        }
    }).then(function() {
        console.log("other function");
        //other function
        cuswriteheader();
        includescript();
        writefooter()
        res.end();   
    });
}


// basic function for reading extension-less URL
// returns a promise whose resolved value is a returnarray data structure
function urlexbasic(filetype, slug) {
    var returnarray = {status:null,contenttype:null,data:null,message:undefined};
    if (slug == undefined) {
        slug = "";
    }

    if (filetype == "none") {
        var filename;
        console.log("path: "+pathdataarray["1"]);
        if (pathdataarray["1"] == "/pre") {
            console.log("substr");
            filename = urlslug.substr(4, urlslug.length-5);
            console.log("1: "+filename);
        }
        urldata(q.pathname, "");
        console.log("slug: "+ slug+" filename: "+ filename);
        if (slug != "" || filename != undefined) {
            console.log("what?")
            if (filename != undefined) {
                filename = "." + filename;
            } else {
                filename = "." + slug;
            }
            console.log("2: "+filename);
        } else {
            filename = "." + urlslug.substr(0, urlslug.length-1);
        }
        console.log("filename: "+ filename);
        return fs.readFileAsync(filename.substr(0, filename.length-1), 'utf8').then(function(data) {
            console.log("readfile");
            var mime = null;
            if (urldataarray.extension == "js") {
                mime = "javascript";
            } else {
                mime = urldataarray.extension;
            }
            console.log("hey running");
            returnarray["status"] = 200;
            returnarray["contenttype"] = 'text/'+mime;
            //res.writeHead(200, {'contenttype': 'text/'+mime});
            returnarray["data"] = data;
            console.log("instatus: "+returnarray["status"]);
            // make returnarray be the resolved value of the promise
            return returnarray;
        }).catch(function(err) {
            console.log("404 error");
            returnarray["status"] = 404;
        });
    } else {
        urldata(q.pathname, filetype);
        var filename;
        if (pathdataarray["1"] == "/pre") {
            console.log("substr");
            filename = urlslug.substr(4, urlslug.length-5);
            console.log("11: "+filename);
        }
        console.log("2slug: "+ slug+" filename: "+ filename);
        if (slug != "" || filename != undefined) {
            if (filename) {
                filename = "." + filename + "." + filetype;
            } else {
                filename = "." + slug +"."+ filetype;
            }
            console.log("22: "+filename);
        } else {
            filename = "." + urlslug.substr(0, urlslug.length-1) + "." + filetype;
        }
        return fs.readFileAsync(filename, 'utf8').then(function(data) {
            var mime = null;
            if (urldataarray.extension == "js") {
                mime = "javascript";
            } else {
                mime = urldataarray.extension;
                console.log("ok");
            };

            if (pathdataarray["1"] == "/pre") {
                returnarray["status"] = 200;
                returnarray["contenttype"] = 'text/plain';
                //res.writeHead(200, {'contenttype': 'text/plain'});
                //res.write("<pre><code>");
                returnarray["data"] = data;
                //res.write("</code></pre>");
            } else {
                returnarray["status"] = 200;
                returnarray["contenttype"] = 'text/'+mime;
                //res.writeHead(200, {'contenttype': 'text/'+mime});
                console.log("pure data");
                returnarray["data"] = data;
            }
            // make returnarray be the resolved value of the promise
            return returnArray;
        }).catch(function(err) {
            return urlex(dataerrcall[filetype]);
        });
    }
}

// returns a promise whose resolved value is a returnArray 
function urlex(filetypeex, slugex) {
    //console.log(urlexbasic(filetypeex, slugex));
    return urlexbasic(filetypeex, slugex).then(function(returnArray) {
        console.log("return: "+returnarray.status);
        responsearray["status"] = returnarray["status"];

        responsearray["contenttype"] = returnarray["contenttype"]
        responsearray["message"] = returnarray["message"];
        responsearray["data"] = returnarray["data"];
        return responsearray;
    });
}

This uses several nice promise features such as chaining (to sequence async operations).

Sorry this is a good solution but since I'm building a custom CMS web app All the function should stand alone and can be used for other use and also let's imagine I need to handle status like `200` and also content-type and for a good code it should be minify to one function and not be write for every time, and also this function should be only called once but not twice if possible. So if there is no other better solution I'll consider you're but if there is solution that will be more simplified with other package, I'll consider them. — Andrew Young, Jul 05 '17 at 14:53
And for the last solution it use `.than(code)` but my case the `fs` is already a callback function if the previous function fails so it's good for others with simple web app but not for me — Andrew Young, Jul 05 '17 at 14:55
@Andrewyg - If you show us your REAL code and your real issue, we can help a lot better and a lot more specifically. This is how you handle asynchronous operations in nodejs. Either of these absolutely will work for you. To continue operation after an asynchronous operation, you HAVE to either put the code inside the callback or you call some function from the callback that contains the code you want to execute or you use async/await in ES7 which is just a different syntax for doing the same thing. Those are your three options, period. That's how node.js works. — jfriend00, Jul 05 '17 at 14:59
@Andrewyg - Asynchronous operations in nodejs are NOT blocking so you have to continue operation in the callback. — jfriend00, Jul 05 '17 at 15:00
@Andrewyg - This is not only for people with a simple web app. This is how all web apps do things in nodejs. This is one of the key principles you have to learn when programming in nodejs. It may seem odd and foreign at the moment, but you get very used to it. And, if you advance to promises, it gets even easier, particular when you have multiple asynchronous operations to sequence or coordinate. — jfriend00, Jul 05 '17 at 15:03
here is my code: https://files.andrewyg.net/code/705.js and I use fs.readFile in urlex function — Andrew Young, Jul 05 '17 at 15:05
@Andrewyg - Please add the code for the function `urlex()`. `urlex` needs to either return a promise or take a callback as an argument that will indicate when it is done and then the calling code can use that to know when its operation is complete and the rest of the code should be executed. — jfriend00, Jul 05 '17 at 15:13
I update it `urlexbasic()` function returns a array since I said it can be use for other like API than `urlex` function assign the value of the object `urlexbasic` return to `responsearray`. — Andrew Young, Jul 05 '17 at 15:20
@Andrewyg - I have to head out to do other things now. I can probably come back to this later in the day. The general idea is still the same that `urlex()` needs to either return a promise that is resolved when all of its operations are done or it needs to take a callback as an argument and call that callback when it is done and then the caller uses those to know when it is done. — jfriend00, Jul 05 '17 at 15:47
@Andrewyg - I wrote a version of your code using promises to sequence asynchronous operations. Please take a look and let me know what you think. Since promises are built into Javascript starting with ES6, they are a great tool for managing and sequencing asynchronous operations. If you're going to be much of this type of development, it would be well worth your while to learn how to use them. — jfriend00, Jul 05 '17 at 22:37

How to let asynchronous readFile method follow order in node.js

code I have issue

2 Answers2

Linked