0

I have a problem with Axios and or Cheerio. I am trying to get all the divs from a web page but i only manage to get the parent divs.

Here is my code:

const axios = require("axios");
const cheerio = require("cheerio");

const slUrl = "https://sl.se/";

axios.get(slUrl).then((response) => {
  const $ = cheerio.load(response.data);
  $("div").each((index, element) => {
    const divClass = $(element).attr("class");
    console.log(divClass);
  });
});

The result of this is:

sl_fragment_header_1-18-1
undefined
sl_fragment_travel-planner_3-12-1
sl_fragment_travel-result_3-9-0
sl_fragment_departures_1-12-0
sl_fragment_traffic-situation-summary_1-4-0
sl_fragment_highlighted-teaser_1-0-4
sl_fragment_teasers_1-1-1
sl_fragment_traffic-news_1-0-0
sl_fragment_footer_1-9-0

These are the classes of the divs that i manage to collect. The problem is that when i visit this site and inspect the content, every one of these divs has child divs.

Here is an example of the div with class 'sl_fragment_travel-planner_3-12-1':

Travel planner div

I cannot manage to get these child divs. What am i missing here?

I forgot to mention that when I run the exact same code on https://www.aftonbladet.se i get all the divs.. Not only the parent divs...

Brewsli
  • 127
  • 1
  • 10

1 Answers1

0

The problem is that $("div") is only selecting the divs on the root level. you need to address the root div and all children divs.

There you should find your solution: select elements and all children

nip
  • 33
  • 1
  • 8
  • Well the problem is that i tried exactly the same code on another page and in that case i got all divs, even the child ones... – Brewsli Oct 21 '21 at 13:06