2

I have some difficulties understanding how the stack works during recursive calls in JS.

I want to make a bot that parses Wikipedia articles, and provides me links of these articles. I'm using the wikimedia API, which allows to fetch the data I'm looking for.

I'm using an object of my own, called Leaf, which allows me to build a tree. Here's its structure:

Leaf = {
"id":123,
"name":foo,
"depth":1,
"children":[Leaf, Leaf, Leaf]
}

Here's how I intend to do the fetching: I start by getting data from a random page, and put it in a Leaf object;

 randomPageFetch = async () => {
    const responseJson = await fetch('https://fr.wikipedia.org/w/api.php?action=query&format=json&rnnamespace=0&list=random&rnlimit=1&origin=*').then(res => res.json())
    let newLeaf = new Leaf (responseJson.query.random[0].id,responseJson.query.random[0].title, [], 1)
    newLeaf = await this.fetchLinksFromPage(newLeaf)
    return newLeaf
  }

I do this once, as this is the only time I need a random page. Once it's done (as you've seen), I call a second function, which parse links in the article.

  fetchLinksFromPage = async (leaf) =>{
    var responseJson = await fetch('https://fr.wikipedia.org/w/api.php?action=query&format=json&gpllimit='+MAX_LINKS_PER_PAGE+'&titles='+leaf.name+'&generator=links&origin=*').then(res => res.json())
    try{
       await Object.entries(responseJson.query.pages).forEach(async(element) => {
        if (element[1].ns === 0 && element[1].pageid > 0){
          var newLeaf = await new Leaf(element[1].pageid, element[1].title, [], leaf.depth + 1)
          if (newLeaf.depth < MAX_DEPTH) {
           newLeaf.children  = await this.fetchLinksFromPage(newLeaf)
          }
          leaf.children.push(newLeaf)
          return leaf
        }
      })
    }
    catch(e){
      console.log(e, "IN RESPONSE   ", responseJson, "FROM   ", leaf.name)
    }
    return leaf
  }

With my understanding of recursivity, it should make a succession of fetch like this:

fetch(depth = 1)
  fetch(depth = 2)
    fetch(depth = 3)
    fetch(depth = 3)
  fetch(depth = 2)
    fetch(depth = 3)
    fetch(depth = 3)
fetch(depth = 1)
  fetch(depth = 2)
    fetch(depth = 3)
    fetch(depth = 3)
  fetch(depth = 2)
    fetch(depth = 3)
    fetch(depth = 3)

However, when I log in the forEach loop, it seems to do something like this

fetch(depth = 1)
fetch(depth = 1)
fetch(depth = 1)
**RETURN TO FIRST FUNCTION**
  fetch(depth = 2)
  fetch(depth = 2)
  fetch(depth = 2)
   **RETURN TO FIRST FUNCTION**
    fetch(depth = 3)
    fetch(depth = 3)
    fetch(depth = 3)

I'm new to async/await, so I'm not sure this behavior is intended. Do you have any ideas on how to fix this issue?

LRobert
  • 23
  • 1
  • 6
  • 1
    async-await does not behave as you think inside high order functions, check this:https://stackoverflow.com/questions/37576685/using-async-await-with-a-foreach-loop. Replace it with a simple for ..of loop and try. Also, you don't need to use then-catch as you are already using async-await while fetching. – Raghu Chahar Jan 22 '20 at 13:57
  • `await [...].forEach()` === `await undefined` and therefore pointless – Thomas Jan 22 '20 at 14:01
  • Thanks a lot, I didn't knew about `for...of` loop, it solves my problem! – LRobert Jan 22 '20 at 14:07

0 Answers0