Edit: Thanks stackoverflow community, this post is clearly a complete duplicate of another post made two years ago. God forbid that a problem could have more than one cause, or something has changed in two whole years. Or that I saw that post and tried everything I saw in it.
It's been a while since we solved the issue so I can't quite remember what exactly solved it, but updating this post might be a good idea since we solved it and others are coming across this post.
Anyway, we figured out that the chromedriver version was wrong, even though we were using the most updated version. It appeared that python begun using the snap store version of chromedriver for seemingly no reason, and we were able to get on the path to fixing it by adding
executable_path='/snap/bin/chromium.chromedriver' to the webdriver options. I don't quite remember if that was exactly what the issue was, but this put us on the right track.
Since we are running the code on VMs, we also ran sudo cp /snap/chromium/1424/usr/lib/chromium-browser/chromedriver /path/to/chromedriver/in/VM/script/file
Original Question:
I have had a python selenium script running smoothly for about a month, and last Thursday (12-10-20) it stopped working for apparently no reason. I am running the script in a virtual env and have the most recent version of the chromedriver webdriver.
My environment is being run on a remote VM with no screen and with everything on the most recent version updated, but specifically: Ubuntu 20.04.1, Python 3.8.5, chromium-chromedriver 1:85.0.4183.83-0ubuntu0.20.04.2 and chromedriver 2.24.1.
I am using a virtual display to run the script.
Previously this script worked with just:
chrome_options = Options()
display = Display(visible=0, size=(800, 800))
display.start()
chrome_options.add_argument("--headless")
chrome_options.add_argument('--no-sandbox')
self.driver = webdriver.Chrome(chrome_options = chrome_options)
self.driver.get(...)
And here is everything that I have added in tried to fix the webdriver:
chrome_options = Options()
display = Display(visible=0, size=(800, 800))
display.start()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--start-maximized")
chrome_options.add_argument("--start-fullscreen")
chrome_options.add_argument("--no-proxy-server")
chrome_options.add_argument("--proxy-server='direct://'")
chrome_options.add_argument("--proxy-bypass-list=*")
chrome_options.add_argument(f'--proxy-server={None}')
chrome_options.add_argument('disable-gpu')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('/scrapr/services/chromedriver')
self.driver = webdriver.Chrome(chrome_options = chrome_options)
self.driver.get(...)
I have also reinstalled all dependencies and downloaded the most recent version chromedriver. As far as I am aware, everything is completely up to date.
This code does work on my laptop, however when I attempt to run the code on the remote VM, I get this error:
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed. (chrome not reachable) (The process started from chrome location /snap/chromium/1424/usr/lib/chromium-browser/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Any ideas?
