2

I am trying to use selenium and headless chrome to scrape a website so since I don't have much experience I tried to test with this simple script but I seem to be getting the error below. Do you think there are any compatibility issues with the drivers?

Code:

from selenium import webdriver

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--window-size=1420,1080')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
outputdir='~/zycus_report_scraper/data_engineering/scripts/python'
service_log_path = "{}/chromedriver.log".format(outputdir)
service_args = ['--verbose']
browser = webdriver.Chrome(chrome_options=chrome_options, executable_path=r'/home/user/.local/bin/chromedriver', service_args=service_args, service_log_path=service_log_path)
browser.get('https://www.twitter.com')
print(browser.page_source.encode('utf-8'))
browser.quit()

Error:

Traceback (most recent call last):
  File "zycus_selenium.py", line 12, in <module>
    browser = webdriver.Chrome(chrome_options=chrome_options, executable_path=r'/home/groballe/.local/bin/chromedriver', service_args=service_args, service_log_path=service_log_path)
  File "/home/groballe/.local/lib/python3.6/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
    self.service.start()
  File "/home/groballe/.local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 98, in start
    self.assert_process_still_running()
  File "/home/groballe/.local/lib/python3.6/site-packages/selenium/webdriver/common/service.py", line 111, in assert_process_still_running
    % (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service /home/user/.local/bin/chromedriver unexpectedly exited. Status code was: 1

Driver versions:

chromedriver version: ChromeDriver 80.0.3987.106 (f68069574609230cf9b635cd784cfb1bf81bb53a-refs/branch-heads/3987@{#882})

chromium-browser version: 80.0.3987.87

selenium: selenium-3.141.0

Edit: This is on a Windows Subsystem for Linux (WSL).

Dominus
  • 29
  • 1
  • 7

0 Answers0