1

I would like to scrape this web page:

this web page

but fiding it hard and impossible. How do I scrape this headers from news, and for each radio button?(bearish and bullish are radio buttons).

I also need to do it for eur and usd (so 4 scrapes from that page). How do or can I do this?

Nothing seems to work, I don't have a lot of knoledge of BeautifulSoap, but if tried. I've tried with css, with classes and id, and always no response. It seems it doesn't find those classes.

this was my last attempt:

for e in html.res.select("[class~=fxs_tab]"):
        print(e.li.text)
        print(type(e.li.text))
        titulares.append(e.li.text)

this is the headers I would like to scrape

Habi
  • 45
  • 6
  • Hi, Javi! I'd suggest taking a look at the following post: https://stackoverflow.com/questions/45643796/print-list-items-in-beautiful-soup-python . This is exactly what you should be looking for. – Muneeb Ahmad Khurram Feb 24 '23 at 16:38

1 Answers1

0

The section that you want to scrape is rendered by JavaScript. You'll have to use a JS rendering service like Selenium in addition to using BeautifulSoup. You can check what HTML BeautifulSoup gets to parse by going to the 'View Page Source' tab in your browser and you'll find out that the div with class fxs_tab doesn't exist and consequently BS4 can't access it.