How to deal with multiple async calls in the Wikipedia API

Question

I want to find all the links contained on a Wikipedia page, but how can I get around the async execution?

The below code fetches a list of page links. That I can do. The problem is if I need to get the second page of links (500 links per request), I don't know how to make it run sequentially. The reason I need it sequentially is I'm wrapping it around a function that gets 'all' the links. I am not sure if this is even good practice.

function findLinksInPage(parent, plcontinue) {
    var url = 'https://en.wikipedia.org/w/api.php?format=json&action=query&prop=links&titles=' + parent + '&pllimit=500'
    if(plcontinue != undefined) {
        url += '&plcontinue=' + plcontinue
    }

    var links = []
    var plcontinue
    fetch(url)
        .then(response => response.json())
        .then(data => {
            if(data.continue != undefined) {
                plcontinue = data.continue.plcontinue
            }
            for(key in data.query.pages) {
                const linksPart = data.query.pages[pageId].links
                linksPart.forEach( value => {
                    const link = value.title
                    // console.log('Link found: ' + link)
                    links.push(link)
                })
                var nextLinks = findLinksInPage(parent, plcontinue)
                links.concat(nextLinks)
                return links
            }
        })

        .catch(error => {
            console.log(console.error(error))
        })
}

Why ".catch"? And the empty line before (e.g., implicit [semicolon problems](https://stackoverflow.com/questions/537632/should-i-use-semicolons-in-javascript))? Is it supposed to work? — Peter Mortensen, Apr 24 '22 at 13:21

score 1 · Answer 1 · edited Apr 24 '22 at 13:11

If you are not limited to some browser version, I would suggest to use the async/await syntax to get it more clear:

async function findLinksInPage(parent, plcontinue) {
    var url = 'https://en.wikipedia.org/w/api.php?format=json&action=query&prop=links&titles=' + parent + '&pllimit=500'
    if(plcontinue != undefined) {
        url += '&plcontinue=' + plcontinue
    }

    var links = []
    var plcontinue

    try {
        let response = await fetch(url);
        let data = await response.json()

        if(data.continue != undefined) {
            plcontinue = data.continue.plcontinue
        }

        for(key in data.query.pages) {
            const linksPart = data.query.pages[pageId].links
            linksPart.forEach( value => {
                const link = value.title
                // console.log('Link found: ' + link)
                links.push(link)
            })
            var nextLinks = await findLinksInPage(parent, plcontinue)
            links.concat(nextLinks)
        }
        return links // <- not inside for
    }
    catch(error) {
        console.error(error)
    }
}

The function findLinksInPage is a promise, so in the first call you can add a findLinksInPage(parent, plcontinue).then(allLinks=> console.log(allLinks)) call.

Why the lone semicolon? – Peter Mortensen Apr 24 '22 at 13:15 — Peter Mortensen, Apr 24 '22 at 13:15

How to deal with multiple async calls in the Wikipedia API

1 Answers1