Increment Counter prints prematurely in Node JS

Question

I am copying 157 documents from 1 collection to another. I just want to count how many documents are inserted or skipped but it always gives wrong output (like only 100 but it should be 157 as the destination column was empty and all of 157 are inserted now). Any help will be heartily appreciated.

var cursor = await collection.find(query).project(projection);
var countInserted = 0;
var countSkipped = 0;

await cursor.forEach(async function(i) {
    i.ts_imported = new Date();
    //console.log(i);
    ret = await EmptyCollection.updateOne(
      { URL: i.sourceURL },
      {
          $setOnInsert: {
              URL: i.sourceURL,
              status: 0,
              remarks: "in the run got 0 jobs",
              collection: col
          }
      },
      { upsert: true }
    );
    if (await ret.upsertedCount == 0) {
        ++countSkipped;
    }
    else {
        ++countInserted;
    }
    //console.log(countInserted); //This shows actual count.
});

console.log(countInserted); //This shows wrong count.
res.writeHead(200, {'Content-Type': 'text/html'});
res.write(countInserted+' rows inserted.<br/>'+countSkipped+" rows skipped.");
res.end();

score 0 · Answer 1 · answered May 27 '20 at 06:33

0

Try the traditional for(;;;) syntax instead of forEach, this might resolve your issue!

answered May 27 '20 at 06:33

Sagar Kulkarni

483
4
15

but that will lead me to infinite loop and wont make any sense to count the skipped documents. @sagar – TRJ May 27 '20 at 06:38
I didn't fill it with conditions you had to fill it yourself, and I didn't hint for infinit loop at all ! – Sagar Kulkarni May 27 '20 at 08:44

score 0 · Answer 2 · answered May 27 '20 at 06:35

None of the array processing functions such as forEach, map, filter and reduce work with async/await. You will need to use either a for loop or a while loop:

       for (x=0; x<cursor.length; x++) {
           let i = cursor[x];
            i.ts_imported = new Date();
            //console.log(i);
            let ret = await EmptyCollection.updateOne({"URL":i.sourceURL},{$setOnInsert: {"URL": i.sourceURL, "status" :0 , "remarks" :"in the run got 0 jobs", "collection" : col}},{ upsert: true });

            // ...

score 0 · Accepted Answer · answered May 27 '20 at 06:55

Thank you guys for trying to help me and showing interest. But I found the answer myself. I just had to use "Promise" to make it happen like below-


    var cursor = await collection.find(query).project(projection);
                var countInserted = 0;
                var countSkipped = 0;
                var arr = await cursor.toArray();
                await Promise.all(
                    arr.map(async i => {
                        ret = await EmptyCollection.updateOne({"URL":i.sourceURL},{$setOnInsert: {"URL": i.sourceURL, "status" :0 , "remarks" :"in the run got 0 jobs", "collection" : col}},{ upsert: true });

                        if(await ret.upsertedCount==0){
                            ++countSkipped;
                        }
                        else {
                            ++countInserted;
                        }
                    })
                );

Increment Counter prints prematurely in Node JS

3 Answers3