I wrote a python script that uses selenium to extract some information on the web and write it to a file. The file runs on a remote server and is executed periodically (every minute from 8:00 to 16:59) with making use of crontab. Everything seems to be working until 16:15. From then every time the script is executed from the cronjob I get the following error when initializing the webdriver:
Message: unknown error: Chrome failed to start: crashed.
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
The code that led up to the crash is
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from datetime import datetime
options = Options()
options.add_argument('--no-sandbox')
options.add_argument("--disable-dev-shm-usage")
options.add_argument('--window-size=1420,1080')
options.headless = True
CHROMEDRIVER_PATH = '/usr/bin/chromedriver'
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)
Searching in the web it has been suggested that it is a problem with resources (and that's why I added the --disable-dev-shm-usage option. And true, if I reboot the server it works again. However, this Exception only occurs when the script is started by the cronjob (after the cronjob worked flawlessly for many interations), when I run the scrip manually the Exception does not occur.
What could be the cause of this, and how do I stop it from happening?
