Run multiple functions in succession while waiting for previous one to complete

Question

I have a script which involves data processing and then uploading/downloading files in mass.

I've been trying to find different ways to sort out an issue I'm having and I've decided I want to try something like this (it appears to be the easiest solution), but I'm not sure if it's achievable with JavaScript.

I'm not that great with async and I have no clue what to do.

Thanks in advance, I've left a vague idea of what I'm doing below.

function one() {
>> do something | data processing
};
function two() {
>> do something | file upload/download
};
function thr() {
>> do something | process two arrays for an output that will be pushed to a global var
};

one();//wait for one to finish
two();//wait for two to finish
thr();//wait for thr to finish

function two() {
    return new Promise( async (resolve, reject) => {
        fs.createReadStream('data.csv')
            .pipe(csv())
            .on('data', (row) => {
                if (row[removed] === '') return;
                const
                        url1        = row[removed],
                        url2        = row[removed]),
                        https       = require('https'),
                        id          = row[removed]);
                let     upload      = [];


                const fetchRealUrl = request(url1, (e, response) => {//need to await completion
                    const FILENAMEREMOVED1= fs.createWriteStream(`${id}-FILENAMEREMOVED1`),
                    downloadRealFile = https.get(response.request.uri.href, function(response) {
                        response.pipe(FILENAMEREMOVED1);
                        console.log(`FILENAMEREMOVED file ${id} downloaded.`);
                    });
                }),
                fetchRealUrl2      = request(url2, (e, response) => {//need to await completion
                    const FILENAMEREMOVED2= fs.createWriteStream(`${id}-FILENAMEREMOVED2`),//need to extract into funciton instead of repeating
                    downloadRealFile2= https.get(response.request.uri.href, function(response) {
                        response.pipe(FILENAMEREMOVED2);
                        console.log(`FILENAMEREMOVEDfile ${id} downloaded.`);
                    });
                });
                //getData(url);
                upload.push(`{"id":"${id}",REMOVED}`);

            })
            .on('end',  (row) => {
                console.info('Completed .');
                resolve(upload);//completes while downloads still running
            });
    });
}

Since you tagged this with `async`/`await`, it seems like you actually know what you should be using. Please show us your attempt at using it in more detail - what did not work when you tried? — Bergi, Nov 20 '19 at 00:39
javascript isn't multithreaded, to get a similar effect you need to break work up into units that then interleave with other units using async/await. — Keith Nicholas, Nov 20 '19 at 00:42
I'm not trying to do multiple things at the same time, I want to do one thing at the same time in order. — Brendan Jennings, Nov 20 '19 at 00:47
@Bergi I want to attempt to try this, I don't really know how to do it though. I'm not great with async but I do know what it is — Brendan Jennings, Nov 20 '19 at 00:48
@BrendanJennings Maybe have a look at my [rules of thumb for working with promises](https://stackoverflow.com/a/25756564/1048572)? Replace "call `then`" with "use `await`". — Bergi, Nov 20 '19 at 00:52

Ryan Kavanagh · Accepted Answer · 2019-11-20T17:10:07.657

1

Try looking at Promises.

function one() {
    return new Promise((resolve, reject) => { 
        //Do something...
        let x = 10
        resolve(x) 
    }
}

function two() {
    return new Promise((resolve, reject) => {
        //Do something else...
        let y = 20
        resolve(y) 
    }
}

one().then(x => { //The value resolved in one() is passed here

    //then() is executed only when one() resolves its promise

    return two()

}).then(y => { //The value resolved in two() is passed here

    //etc...

})

edited Nov 20 '19 at 17:10

answered Nov 20 '19 at 00:55

Ryan Kavanagh

28
5

Don't forget to `return` the inner promises from that chain though, or even just [unnest it](https://stackoverflow.com/a/22000931/1048572). – Bergi Nov 20 '19 at 01:00
You may have saved my ass, I'll give this a go now! – Brendan Jennings Nov 20 '19 at 12:46
Tried this, didn't work. two().then(... still runs before one().then(... has finished – Brendan Jennings Nov 20 '19 at 13:04
Oh my god, thank you! I'll go out and learn how this all works, thanks for your help! – Brendan Jennings Nov 20 '19 at 13:36
You should chain the promises passing a new promise in each return statement rather than nesting them like that. ```one().then(x => { return two()}).then(y => { //etc })``` – Ethan Harris Nov 20 '19 at 14:30
Updated the answer to remove nesting. – Ryan Kavanagh Nov 20 '19 at 17:12
Ok, I have a similar issue I guess, I'm executing what would be two (where some files are downloaded) but the files are still downloaded before we get to three because it resolves before the files are downloaded. Updated the question with code – Brendan Jennings Nov 20 '19 at 17:22

Ethan Harris · Answer 2 · 2019-11-20T03:12:33.553

1

const func1 = new Promise(res =>{
 setTimeout(() => {
     //do some asynchronous work
            res()
 }, 1000)
})

const func2 = new Promise(res =>{
 setTimeout(() => {
     //do some asynchronous work
            res()
 }, 1000)
})


//To run async functions in a waterfall pattern:
func1()
.then(resultOfFunc1 => {
 //do something with resultOfFunc1
 return func2()
})
.then(resultOfFunc2 => {
 //do something with resultOfFunc2
})

//To run async functions in a parallel pattern:
let promiseFunctions = [func1(), func2()]
let result = Promise.all(promiseFunctions)

//result will be an array of resolved promises

edited Nov 20 '19 at 03:12

answered Nov 20 '19 at 01:07

Ethan Harris

1,273
9
13

Don't make it that complicated with `map` though. Just use `const promises = [func1(), func2()]; const result = Promise.all(promises)` – Bergi Nov 20 '19 at 01:43

Run multiple functions in succession while waiting for previous one to complete

2 Answers2