1

I need to build a Python scraper to scrape data from a website where content is only displayed after a user clicks a link bound with a Javascript onclick function, and the page is not reloaded. I've looked into Selenium in order to do this and played around with it a bit, and it seems Selenium opens a new Firefox web browser everytime I instantiate a driver:

 >>> driver = webdriver.Firefox()

Is this open browser required, or is there a way to get rid of it? I'm asking because the scraper is potentially part of a web app, and I'm afraid if multiple users start using it, I will have a bunch of browser windows open on my server.

rottentomato56
  • 1,059
  • 3
  • 13
  • 18
  • 1
    related: [Selenium - Can I hide the browser?](http://stackoverflow.com/a/8910326/4279). See also [Ghost.py](http://jeanphix.me/Ghost.py/) – jfs Mar 02 '13 at 22:18
  • possible duplicate of [Selenium - Can I hide the browser?](http://stackoverflow.com/questions/1418082/selenium-can-i-hide-the-browser) – Petr Janeček Mar 02 '13 at 23:27

1 Answers1

0

Yes, selenium automates web browsers. You can add this at the bottom of your python code to make sure the browser is closed at the end:

driver.quit()
rcdsystems
  • 1,528
  • 1
  • 10
  • 11