Unable to understand asynchronous functions in NodeJS

Question

I have spent 6 hours already trying to figure out how to do this in NodeJS.

I am using NodeJS with Express and MongoDB.

I have a database that has two collections viz. Listings and Categories. Each listing has a "category" which is an ID that maps to a category inside the Categories collection.

What I want to do is, fetch all the listings, then loop through all of them, and get the category title from the Categories collection using the category id.

This is what I had for getting all the listings:

const listings_raw = [];

await db.collection("listings").find().forEach(listing => listings_raw.push(listing));

The code above works fine. The trouble I am having is with this:

const listings = [];

listings_raw.forEach(async listing => {
    const category_id = listing.category;

    const category_title = await db.collection('categories').findOne({_id: objectId(category_id)});

    listing.category_title = category_title;

    listings.push(listing);
});

response.send(listings);

The response.send(listings); gets executed before the listings_raw.forEach completes.

I want to send the data only after all listings have been populated with the category titles.

After spending 6 hours, I have come up with the following hack which I am sure is nasty!

const listings_raw = [];

const em = new events.EventEmitter();

await db.collection("listings").find().forEach(listing => listings_raw.push(listing));

const listings = [];

let counter = 0;

listings_raw.forEach(async (listing) => {
    const category_id = listing.category;

    const category = await db.collection('categories').findOne({_id: objectId(category_id)});

    listing.category_title = category.title;

    listings.push(listing);

    if (counter === listings_raw.length - 1) {
        em.emit('listings:processing:done');
    }

    counter++;
});

em.on('listings:processing:done', () => {
    response.send(listings);
});

Please, can someone explain or guide me on how this should be done in JavaScript?

Basically, I am not able to figure out how to know if all promises have been resolved or not.

Thank you!

https://stackoverflow.com/questions/37576685/using-async-await-with-a-foreach-loop — epascarello, Sep 27 '22 at 15:18

score 5 · Accepted Answer · edited Sep 27 '22 at 15:16

5

The listings_raw.forEach function executes synchronously on the array, even though you are then performing an asynchronous operation within that.

Promise.all will allow you to await for the result of an array of promises. Therefore you can .map the listings to an array of promises which return the updated listing.

const listings = await Promise.all(listings_raw.map(async listing => {
    const category_id = listing.category;

    const category_title = await db.collection('categories').findOne({_id: dependencies.objectId(category_id)});

    listing.category_title = category_title;

    return listing;
});

response.send(listings);

edited Sep 27 '22 at 15:16

Wyck

10,311
6
39
60

answered Sep 27 '22 at 15:13

Joey Ciechanowicz

3,345
3
24
48

1

WOW! You nailed it in what, 1 minute? Do you know what I should be reading more about to be able to understand this very well? My only experience has been with languages like PHP. – Naif Sep 27 '22 at 15:18
1

It's an easy mistake to make until you've nailed how async JS works. MDN has some excellent resources - https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous – Joey Ciechanowicz Sep 27 '22 at 15:19
1

Having a strong understanding of callbacks and promises helps a lot. async-await just makes it look nicer. – thgaskell Sep 27 '22 at 15:23

Unable to understand asynchronous functions in NodeJS

1 Answers1