1

I'm trying to run a scraper using Selenium driver

My py file is (just start)

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException

# Set the url we want
<and so on>

However I get the following error:

Traceback (most recent call last):
  File "main.py", line 37, in <module>
    driver = webdriver.Chrome(chrome_options=options)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
    desired_capabilities=desired_capabilities)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_respons
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming  has crashed.)
  (Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-140-generic

Searching for this error, it seems as if its incompatibility in ChromeDriver and Google-Chrome and Selenium however my Selenium version is 3.141.0, my chrome version is 72.0.3626.119 and my ChromeDriver is 2.46.

So they are all latest and they all seem compatible.

Any ideas what wrong?

Alludy1962
  • 55
  • 6
  • Does this error occur every time? Have you tried to rerun the test a couple of times? – Alex Feb 25 '19 at 12:30
  • Yes, tried several times but same error. – Alludy1962 Feb 25 '19 at 12:30
  • Can you try to run the chromedriver on its own? – Alex Feb 25 '19 at 12:34
  • Take a look at this: https://stackoverflow.com/a/53078276/6808714 – Mahrez BenHamad Feb 25 '19 at 12:34
  • did you specify the path of chrome driver – Manoj biroj Feb 25 '19 at 12:35
  • @MaHrezBenHamad Chrome syslink is already present as it says in that post. – Alludy1962 Feb 25 '19 at 12:46
  • Try this @Alludy1962: options = webdriver.ChromeOptions() options.add_argument('--disable-extensions') options.add_argument('--headless') options.add_argument('--disable-gpu') options.add_argument('--no-sandbox') return webdriver.Chrome(chrome_options=options) – Mahrez BenHamad Feb 25 '19 at 12:52
  • @Alludy1962: Also if you find solution for your problem - you still might get that error occasionally. There is a bug in the chromedriver, and it is not fixed yet https://github.com/heroku/heroku-buildpack-google-chrome/issues/46 – Alex Feb 25 '19 at 13:36

1 Answers1

0

I've ran into this error on Ubuntu 18.04 and it could be one of several things that's causing Chrome to crash. Try setting these flags:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--no-sandbox")
options.add_argument("--headless")

driver = webdriver.Chrome("/usr/local/bin/chromedriver", chrome_options=options)

The --headless flag ensures that Chrome starts in headless mode as I assume you don't have a desktop running on your Ubuntu. the --no-sandbox flag turns off the sandboxing in Linux that sometimes causes problems at the start.

Jim Zhou
  • 311
  • 1
  • 10