0

I have to scrape a web page that has a bunch of dates. The problem is that when the page loads, it shows "Wait for a moment" and then after a second or two, it shows dates. The Python requests library reads the html from the webpage and when I search for the class using BeautifulSoup that contains date, it returns 'Wait for a moment' instead of dates. `

html= requests.get(url).text
date_soup = BeautifulSoup(html, 'lxml')
tag = date_soup.find_all(class_='Subhead-heading js-date-range')
print(tag[0])

`

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
Umer Saeed
  • 63
  • 4
  • 1
    I don't think BeautifulSoup executes any Javascript. Consider [requests-html](https://github.com/psf/requests-html). – Jeppe Nov 24 '19 at 11:17
  • `requests` and `BeautifulSoup` can't execute `JavaScript`. You need [Selenium](https://selenium-python.readthedocs.io/) to control web browser which runs `JavaScript` – furas Nov 24 '19 at 11:32

0 Answers0