0

I am trying to web scrape a website, but am unable to load specific elements of the website because they require javascript to load. I have tried selenium, but it is too slow for what I need to do and requires too much time to process. I have written my code below. Thanks for any help in advance!

Edit: The other question does not solve this question.

Code:

def webScrape():
    url = 'https://www.example.com/'
    r = requests.get(url)
    soup = bs4.BeautifulSoup(r.text, 'html.parser')
    element = soup.find_all('div', attrs={'class': 'main-element'})
    return element
Python12492
  • 21
  • 6
  • If you want the javascript to be parsed, you have to use selenium. Requests will only give you the HTML of the webpage, including links to js files or embedded js code. – Alex K Nov 17 '22 at 15:23

0 Answers0