0

I don't entirely know what I'm doing, so sorry if this is a dumb question, but I've tried to fix this, and I haven't made progress on this damn async. I'm trying to make a scraper to make my own word frequency list, just for the hell of it, and I'm trying to collect links to pages off of pages. Here's the revised code:

async function linkChain (depth, already) {
    if (!depth) { return; }
    let promises = [];
    let length = links.length;
    for (already; already < length; already++) {
        promises.push(
            axios.get(www + links[already]).then(response => {
                // (some stuff where I push to links)
                console.log(links.length); // (It's working)
            })
        );
    }
    Promise.all(promises).then(() => {
        linkChain(depth - 1, already);
    });
}

async function run () {
    await linkChain(2, 0);
    console.log("here");
}

run();

The cheerio and axios stuff is fixed and working. Links is growing close to a million. But before links.length is console.logged, the console logs "here".

I've tried using .then.catch instead of await and async, I've tried a lot of stuff that was dumb, I've tried searching Stack Overflow (although I may have missed a solution, I'm not familiar with the website).

I have removed some extraneous returns; I thought they might help, and when they didn't I tried something else and forgot to remove them. Some of the dumb stuff.

Robert
  • 11
  • 1

1 Answers1

1

linkChain doesn't return anything - and you've marked it async without ever awaiting anything which is kinda pointless.

So, either await Promise.all or return it (without the async keyword). You should also make sure that you return the inner linkChain call by either using return statement or removing the braces.

1.

async function linkChain (depth, already) {
...
   await Promise.all(promises).then(() => {
       return linkChain(depth - 1, already);
   });
}

or

async function linkChain (depth, already) {
    ...
    await Promise.all(promises).then(() => linkChain(depth - 1, already));
}
function linkChain (depth, already) {
   ...
   return Promise.all(promises).then(() => {
        return linkChain(depth - 1, already);
    });
}

or

function linkChain (depth, already) {
   ...
   return Promise.all(promises).then(() => linkChain(depth - 1, already));
}
Jamiec
  • 133,658
  • 13
  • 134
  • 193