I'm trying to run a function inside an axios call, however, the res.json is being called before the function can finish and the result is empty. I logged in the function, and I'm sure that everything is running fine, I do see what I need in the console, but not in the res.json. I believe the main function should be enough, as the other ones are working properly and are very veeery long. In short, they update the global variable articles.
The code follows:
Updated code
const newspapers = [{
"name": "CNN",
"address": "https://edition.cnn.com/specials/world/cnn-climate",
"base": "https://edition.cnn.com"
},
{
"name": "The Guardian",
"address": "https://www.theguardian.com/environment/climate-crisis",
"base": "https://www.theguardian.com"
}, etc...]
const articles = [];
app.get("/news", (req, res) => {
// Query String
const query = checkForQuery(req);
const wordsToSearch = query ? verifyQuery(query) : "";
Promise.all(
newspapers.map(({ name, address, base }) =>
axios
.get(address, {
headers: { "Accept-Encoding": "gzip,deflate,compress" },
})
.then((res) => {
const html = res.data;
console.log({ name, address, base });
// Logs correctly all newspapers, one by one, but the two first articles from the first newspaper is logged after
getElementsCheerio(html, base, name, wordsToSearch);
return articles;
})
)
).then((articles) => {
res.json(articles);
});
});
I'm wondering if I should output res.json() somewhere else, or if async/await would help somehow, but I tried different combinations of the latter and nothing works.
So the question is, how can I make axios wait for that function inside .then() to finish before moving ahead and returning the empty res.json(). Actually, I'm not even sure that's axios fault, it's probably mine. (:
Update: I've adapted the code to run with map instead of forEach. Although I do get some response now, it's only 2 news from the same newspaper repeatedly. I logged the map info, and it's indeed going through all newspaper objects. Maybe I'm returning something wrong somewhere? I don't understand :(