0

I've been trying to create a web scraper which needs to process links on a page and then push them to an array. However, it is an async function (as required by axios) so it can't push to the array before some other code tries to process the array, which has not been filled by the async function yet (and is blank). How would I solve this?

Here is my code currently:

findLinks().then((array) => { // findLinks is my other function which is also async and finds the links for processing
  const paperPages = []; // The array which is meant to store the processed links
  array.forEach((e) => {
    getPapersLink(e)
      .then((pages) => {
        if(typeof pages === "undefined") return
        paperPages.push(pages.replace("fingerprints", "publications")); // This line is meant to push the links to the "paperPages" array after we've processed them (replaced some stuff)
      })
      .then(() => { // Test function, just logs the array. Since this is in a foreach loop, the array is continuously logged to the console
        console.log(paperPages);
        console.log("========================================")
      });
  });
});

// other code to handle the array which is outputted by the above code is meant to be here
// however :
console.log(paperPages);
// outputs "[]" (without the quotes), the original blank array

I can't chain more items to the .then(), as it would simply repeat over and over.

Help is appreciated!

Jim
  • 1
  • 3

0 Answers0