Javascript loop promises synchronously with additional parameter

Question

I have an array of promises I try to process one after the others

function asyncInsertMagFromFile(file, resolve) {
            //some asynchronous processing with file
            resolve();
          }

let magRequests = fs.readdirSync(storeDir).map(function(file) {
            return new Promise(resolve => {
              asyncInsertMagFromFile(file, resolve);
            });
          });

I'm trying to process these promises (magRequests) "synchronously" without success. Lot's of questions about that subject exist but I havent found one that fit with my problem. I tried some of trincot solutions here https://stackoverflow.com/a/40329190/7531001 but I don't know how to handle my file parameter.

For example this doesn't work (await is only valid in async function)

(async function loop() {
            fs.readdirSync(storeDir).forEach(function(file) {
              await new Promise(resolve =>
                asyncInsertMagFromFile(file, resolve)
              );
            });

          })();

Any idea how I can fix this ?

IIRC array functions don't work well for promises, ie they don't wait for them to resolve. On the other hand, normal control flow operations (`for`, `for ... of`, `while`, etc.) should give you more success. — Azami, Feb 20 '19 at 14:38

score 1 · Answer 1 · answered Feb 20 '19 at 14:39

1

You can try to use array.reduce to get a previous value (list of promises) and await it's completion before doing another call of the asyncInserMagFromFile function

(function loop() {
  fs.readdirSync(storeDir).reduce(async (previousPromise, file) => {
    await previousPromise;
    return new Promise(resolve =>
      asyncInsertMagFromFile(file, resolve);
    );
  },Promise.resolve());
})();

answered Feb 20 '19 at 14:39

Krzysztof Krzeszewski

5,912
2
17
30

I like the reduce idea in combination with chaining promises: `fs.readdirSync(storeDir).reduce((previousPromise, file) => { return previousPromise.then( ((file) => () => new Promise(resolve => asyncInsertMagFromFile(file, resolve))(file) ); }, Promise.resolve());` – Gerry Feb 20 '19 at 15:35

score 1 · Answer 2 · answered Feb 20 '19 at 14:42

1

await is only valid in async function

Use .map instead of .forEach, and pass it an async function as a callback.

answered Feb 20 '19 at 14:42

gomes

681
1
7
20

Gerry · Accepted Answer · 2019-02-20T15:19:22.317

You can chain some 'then's one after another.

function asyncInsertMagFromFile(file) {
    return new Promise((resolve, reject) => {
        //some asynchronous processing with file
        // call resolve when finished
        resolve();
    })
}

let chain = Promise.resolve();

fs.readdirSync(storeDir).forEach(function (file) {
    chain = chain.then(((file) => () => asyncInsertMagFromFile(file))(file));
});

chain.then(() => {
    //finished
}).catch((err) => {
   //error in chain
})

Another way would be to use a for statement as mentioned in the comments:

function asyncInsertMagFromFile(file) {
    return new Promise((resolve, reject) => {
        //some asynchronous processing with file
        // call resolve when finished
        resolve();
    })
}

(async () => {
    const files = fs.readdirSync(storeDir);

    for (let i = 0; i < files.length; i++) {
        await asyncInsertMagFromFile(files[i]);
    }
})()

Thanks everybody, after testing them all these answers work great but this is the one I used. Feel free to tell me if another answer is supposed to be the "right" one (even if there are many ways to do the same thing). — Thomas Fournet, Feb 20 '19 at 15:49

score 1 · Answer 4 · answered Feb 20 '19 at 15:11

This may help you;

async function asyncInsertMagFromFile(file) {
 // use await to gather results from processing or return a promise
 return result;
}

function mapRequests(storeDir) {
  return fs.readdirSync(storeDir).map(async function(file) {
    return await asyncInsertMagFromFile(file);
  });
}

mapRequests(storeDir).then((mapped) => {});

The reason your example didn't work is because the forEach handler function uses await but is not declared async.

score 0 · Answer 5 · answered Feb 20 '19 at 15:55

If you want to process in series and process the result then you could use recursion:

const FAILED = {};
const processImages = (files) => {
  const recur=(files,result) => {
    if(files.length===0){
      return result;
    }
    return asyncInsertMagFromFile(files[0])
    .catch(error=>FAILED)//always resolve, place FAILED values when rejected
    .then(success=>recur(files.slice(1),result.concat(success)))
  }
  return recur(files,[])
}

processImages(fs.readdirSync(storeDir)).then(results=>{
  console.log('failed',results.filter(r=>r===FAILED).length)
});

For parallel processing you can use Promise.all:

Promise.all(
  fs.readdirSync(storeDir).map((file)=> asyncInsertMagFromFile(file).catch(FAILED))
).then(results=>{
  console.log('failed',results.filter(r=>r===FAILED).length)
})

Javascript loop promises synchronously with additional parameter

5 Answers5