0

I'm trying to run a function inside an axios call, however, the res.json is being called before the function can finish and the result is empty. I logged in the function, and I'm sure that everything is running fine, I do see what I need in the console, but not in the res.json. I believe the main function should be enough, as the other ones are working properly and are very veeery long. In short, they update the global variable articles.

The code follows:

Updated code

const newspapers = [{
    "name": "CNN",
    "address": "https://edition.cnn.com/specials/world/cnn-climate",
    "base": "https://edition.cnn.com"
  },
  {
    "name": "The Guardian",
    "address": "https://www.theguardian.com/environment/climate-crisis",
    "base": "https://www.theguardian.com"
  }, etc...]
const articles = [];

app.get("/news", (req, res) => {
  // Query String
  const query = checkForQuery(req);
  const wordsToSearch = query ? verifyQuery(query) : "";

  Promise.all(
    newspapers.map(({ name, address, base }) =>
      axios
        .get(address, {
          headers: { "Accept-Encoding": "gzip,deflate,compress" },
        })
        .then((res) => {
          const html = res.data;
          console.log({ name, address, base });
// Logs correctly all newspapers, one by one, but the two first articles from the first newspaper is logged after

          getElementsCheerio(html, base, name, wordsToSearch);
          return articles;
        })
    )
  ).then((articles) => {
    res.json(articles);
  });
});

I'm wondering if I should output res.json() somewhere else, or if async/await would help somehow, but I tried different combinations of the latter and nothing works.

So the question is, how can I make axios wait for that function inside .then() to finish before moving ahead and returning the empty res.json(). Actually, I'm not even sure that's axios fault, it's probably mine. (:

Update: I've adapted the code to run with map instead of forEach. Although I do get some response now, it's only 2 news from the same newspaper repeatedly. I logged the map info, and it's indeed going through all newspaper objects. Maybe I'm returning something wrong somewhere? I don't understand :(

Bella
  • 414
  • 2
  • 13
  • You can't wait. Asynchronous programming does not work that way. – Pointy Dec 17 '22 at 23:20
  • That's all? It's impossible to do what I want to do? I don't believe that '-' – Bella Dec 17 '22 at 23:24
  • There are several dozen answers here that show how to make this work. Search for "forEach promise asynchronous loop" and see what you find. – jfriend00 Dec 17 '22 at 23:27
  • 1
    Of course you can program the code to do what you want, just not with `.forEach`, which is a **synchronous** looping method which discards any return value from the function it calls without waiting for any promises created in the function to settle. – traktor Dec 17 '22 at 23:53
  • @Bella, If you use `puppeteer`, you can search news by sequential order. example, search `abc` keyword from news, got result, next search `def` keyword got the 2nd results. – Bench Vue Dec 18 '22 at 02:43
  • Thank you @BenchVue, but I don't want to use a library here, I want to learn how to do it myself :) – Bella Dec 18 '22 at 10:26
  • @Bella, can you make new question? I will add an answer. I can't replay here due to closed question. Let me know you create it via this channel. – Bench Vue Dec 18 '22 at 11:56
  • Just posted a new question :) https://stackoverflow.com/questions/74841317/axios-response-loop-through-function-returning-only-the-result-of-first-mapped-o Thank you so much S2 – Bella Dec 18 '22 at 12:30
  • @Bella Did you got it or still have a question for sequential keyword search? – Bench Vue Dec 18 '22 at 12:50
  • Yes, it works now. I'm so happy *-* Can't believe I got this far :D thank you for the support :) – Bella Dec 18 '22 at 12:57
  • OK, good to hear you happy. – Bench Vue Dec 18 '22 at 13:00

0 Answers0