I am trying to use recursive promises to fetch all children categories inside categories of scraped website. It seems that my code should work just fine but it only works if i use "reject" instead resolve. Somethings stopping promise from beeing fullfilled but i cannot manage to find the problem cause.
export async function ScrapeCategory(category) {
return new Promise(async (reject, resolve) => {
let promises = [];
var i = 1;
var ilen = category.pageCount;
while (i <= ilen) {
var res = await GetResponse(baseURL + category.link + "?pageId=" + i);
var $ = cheerio.load(res.data);
let productTiles = $(".product-link-ui");
var y = 0;
var ylen = productTiles.length;
while (y < ylen) {
promises.push(await FetchProducts(productTiles[y], category, i));
y++;
}
i++;
}
await Promise.all(promises).then(async (prodResult) => {
category.products = prodResult;
if (category.children) {
let childPromises = [];
var z = 0;
var zlen = category.children.length - 1;
while (z < zlen) {
childPromises.push(await ScrapeCategory(category.children[z]));
z++;
}
await Promise.all(childPromises).then((childResults) => {
console.log(childResults);
category.children = childResults;
});
}
});
resolve(category);
});
}
This is the promise that i try to fulfill. I appreaciate any kind of help.
It should work in that way that first it fetches products for the category, than when it is fulfilled it checks if category have child categories - if yes than it starts to fetch those categories. In the end it resolves main category with child categories inside as well.
Best regards
I tried to move code outside "then" function of promise all statement but there was no change. Problem seems to be with recursive statements that are never fullfilled (comes with undefined as return).
When i change resolve to reject the code works fine but it bothers me cause i know something is wrong and it cannot be the right sollution.
