0

I used beautifulsoup and requests to go to a website and scrape some info. I plan to use it alongside selenium to log in to a website. But there's a series of randomly generated numbers that appear everytime you visit the site. And since selenium and beautifulsoup create two different instances of the site( selenium opens the browser and goes to the url from there while beautiful visits the website to scrape it's info directly through the console through requests module), so the numbers are different in the two instances. I want to go to the site using the webdriver from selenium, and then scrape the info of that specific page using beautifulsoup or any other module. Is it possible?

I want to automate the 'log in' process of this website: http://www.educationboardresults.gov.bd/

VTorch
  • 21
  • 1
  • What's stopping you from parsing HTML returned by Selenium? More on how to do that [here](https://stackoverflow.com/questions/7263824/get-html-source-of-webelement-in-selenium-webdriver-using-python). – ARentalTV Jun 14 '19 at 08:00
  • yes you can do that. when you use selenium, store the html source, ie `html = driver.page_source`, then feed that into BS ie. `soup = BeautifulSoup(html, 'html.parser'` – chitown88 Jun 14 '19 at 08:07

0 Answers0