2

I'm trying to scrape Google Chrome Extension Store reviews using Selenium and BeautifulSoup. However, I can't seem to get Selenium up and running, even with the latest version of Chromedriver, which seemed to solve the problem for other people asking about the same thing on this site. A blank Chrome window will briefly open, then close. Here's the traceback I get:

Traceback (most recent call last):
File "scrape_chrome_reviews.py", line 5, in <module>
driver = webdriver.Chrome(chromedriver)
File "/Users/serenity/anaconda/lib/python2.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 69, in __init__
desired_capabilities=desired_capabilities)
File "/Users/serenity/anaconda/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 92, in __init__
self.start_session(desired_capabilities, browser_profile)
File "/Users/serenity/anaconda/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 179, in start_session
response = self.execute(Command.NEW_SESSION, capabilities)
File "/Users/serenity/anaconda/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 234, in execute
response = self.command_executor.execute(driver_command, params)
File "/Users/serenity/anaconda/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 408, in execute
return self._request(command_info[0], url, body=data)
File "/Users/serenity/anaconda/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 440, in _request
resp = self._conn.getresponse()
File "/Users/serenity/anaconda/lib/python2.7/httplib.py", line 1132, in getresponse
response.begin()
File "/Users/serenity/anaconda/lib/python2.7/httplib.py", line 453, in begin
version, status, reason = self._read_status()
File "/Users/serenity/anaconda/lib/python2.7/httplib.py", line 417, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine: ''

and the code I'm running is just this:

from selenium import webdriver                                              
chromedriver = '/usr/local/Cellar/chromedriver/2.27/bin/chromedriver'
driver = webdriver.Chrome(chromedriver)
driver.quit()

Any suggestions? I'm really stumped.

cetaceanNeeded
  • 75
  • 7

1 Answers1

1

Dowload the latest version of chromedriver and place it to somewhere in the project and call it instead of the on bin file.

from selenium import webdriver 
driver = webdriver.Chrome("/path/to/chromedriver")
Mesut GUNES
  • 7,089
  • 2
  • 32
  • 49
  • got a different error: `Exception AttributeError: "'Service' object has no attribute 'process'" in > ignored` – cetaceanNeeded Jan 11 '17 at 21:58
  • @cetaceanNeeded use latest version of chromedriver instead of the one in bin folder. check updated answer. – Mesut GUNES Jan 11 '17 at 22:05
  • so I actually copied the latest version of chromedriver to that directory because that filepath is the only one that works properly. any other filepath besides the one homebrew originally installed to throws an error. – cetaceanNeeded Jan 11 '17 at 22:07
  • @cetaceanNeeded seems like chromedriver 2.25 solves this problem. Try that version – Mesut GUNES Jan 11 '17 at 22:11
  • that throws this error: `File "/Users/serenity/anaconda/lib/python2.7/site-packages/selenium/webdriver/common/service.py", line 81, in start os.path.basename(self.path), self.start_error_message) selenium.common.exceptions.WebDriverException: Message: 'chromedriver_3' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home Exception AttributeError: "'Service' object has no attribute 'process'" in > ignored` – cetaceanNeeded Jan 11 '17 at 22:16
  • the problem is related to path, check this answer: http://stackoverflow.com/a/29858817/2568849 – Mesut GUNES Jan 11 '17 at 22:21