Retry asynchronous operations that were unsuccessfull - How to avoid await inside of loops (no-await-in-loop) in EsLint?

Question

I have an array of tours from JSON and import all of them into a database.

I use for-loop because I want to extract all the error tours into another file so I can fix it and re-import again later.

Here is my code, and it works as expected.


const importData = async () => {


  const tours = JSON.parse(
    await fs.readFile(path.join(__dirname, 'data', 'final.json'), 'utf8')
  );
 

  const errorTours = [];


  for (let i = 0; i < tours.length; i += 1) {

   const tour = tours[parseInt(i, 10)];

     try {
       await Tour.create(tour);
     } catch (e) {
       errorTours.push({ tour, error: e });
     }
  }


  await fs.writeFile('errorTour.json', JSON.stringify(errorTours));

  console.log('finished!!! :tada:');
}

but I got "Disallow await inside of loops (no-await-in-loop)" EsLint Error.

Performing an operation on each element of an iterable is a common task. However, performing an await as part of each operation is an indication that the program is not taking full advantage of the parallelization benefits of async/await.

Usually, the code should be refactored to create all the promises at once, then get access to the results using Promise.all(). Otherwise, each successive operation will not start until the previous one has completed.

Maybe in my case, the Promise.allSettled() suit better, right ?

I'm new in JS and quite confused about how to change my async await code to Promise code to using Promise.allSettled.

Or is there any better way to retry asynchronous operations that were unsuccessful?

Can you guys show me the direction in this case?

Thanks,

Can you push your promises to an array and then return `promise.all([promises])`? — Atlante Avila, Aug 07 '20 at 03:20
https://stackoverflow.com/questions/37576685/using-async-await-with-a-foreach-loop — codemax, Aug 07 '20 at 03:24
Why are you calling `parseInt(i,10)`? `i` is already an integer as you just declared and assigned it yourself. — jfriend00, Aug 07 '20 at 06:39
@jfriend00 I used [Eslint security](https://github.com/nodesecurity/eslint-plugin-security) and it show up this error: Variable Assigned to Object Injection Sinkeslint(security/detect-object-injection) I follow what this topic to fix that: https://stackoverflow.com/questions/51715616/how-to-fix-codacy-alert-generic-object-injection-sink — Thinh NV, Aug 07 '20 at 11:58
@ThinhNV - That's a bogus warning in this case that caused you to add unnecessary code. See https://security.stackexchange.com/questions/170648/variable-assigned-to-object-injection-sink-security-detect-object-injection. Since `i` comes directly from your code here, there is no injection risk at all. — jfriend00, Aug 07 '20 at 14:53
thank @jfriend00, I will try to report that issue to the maintainer of that plugin. ps: look like the problem still stand, many people talk about it : https://github.com/nodesecurity/eslint-plugin-security/issues/21 Using Lint plugins seems like using a double-edged sword, it can help us a lot and it can hurt us a lot with too much false positives :-| — Thinh NV, Aug 07 '20 at 15:13
You have to learn what the warning means and whether it is actually appropriate for your code or not. And, if not, then you can either turn off that specific warning or insert a comment that tells it to ignore that warning in this part of the code. You can't/shouldn't use these tools without understanding what the warnings actually mean that they provide and whether they are indeed relevant or not. — jfriend00, Aug 07 '20 at 17:02

Aakash · Accepted Answer · 2020-08-09T02:40:58.767

1

How about this?

 const TourPromises = tours.map(tour => Tour
    .create(tour)
    .catch(e => errorTours.push({tour, error: e}))
  )
 await Promise.all(TourPromises);

Good Luck...

edited Aug 09 '20 at 02:40

answered Aug 07 '20 at 04:46

Aakash

21,375
7
100
81

1

@ThinhNV - Just so you know, this does not work exactly the same as your original code. This runs all the asynchronous operations in parallel. Your original code ran them in sequence (one after the other). Parallel may get you results faster, but if your array was larger, it may cause problems by having too many requests in flight at the same time (such as rate limiting by the target host or larger memory usage). – jfriend00 Aug 09 '20 at 15:34

score 0 · Answer 2 · answered Aug 07 '20 at 04:01

Here is my tried, thank @Atlante Avila and Eslint docs

  const TourPromises = [];
  for (let i = 0; i < tours.length; i += 1) {
    const tour = tours[parseInt(i, 10)];

    TourPromises.push(
      Tour.create(tour).catch((e) => {
        errorTours.push({ tour, error: e });
      })
    );
  }

  await Promise.all(TourPromises);

old code: Took 5466.025282999966 milliseconds.

new code: Took 1682.5688519999385 milliseconds.

Look quite better,

Retry asynchronous operations that were unsuccessfull - How to avoid await inside of loops (no-await-in-loop) in EsLint?

2 Answers2