I am facing this issue; I know this question is already present and I have tried the solutions mentioned hence asking this with my configurations and code. I am getting the following error when Celery tasks try to use Selenium;
[2020-06-12 23:23:04,228: WARNING/ForkPoolWorker-1] Message: unknown error: Chrome failed to start: exited abnormally.
(chrome not reachable)
(The process started from chrome location /opt/python/run/venv/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
[2020-06-12 23:23:04,230: ERROR/ForkPoolWorker-1] Task Update All Results[73583ec6-474a-4f20-8d88-d7be0a150642] raised unexpected: NameError("name 'sleep' is not defined",)
Traceback (most recent call last):
File "/opt/python/bundle/42/app/fixtures/models.py", line 1035, in get_flash_result
driver = webdriver.Chrome(options=opts)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/chrome/webdriver.py", line 81, in __init__
desired_capabilities=desired_capabilities)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/opt/python/run/venv/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
(chrome not reachable)
(The process started from chrome location /opt/python/run/venv/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
I have a Django app on an EC2 server via Elastic Beanstalk which runs periodic tasks using Celery however when webdriver is set I get the error.
The Celery task calls a class method on my model which contains the selenium code. The error doesn't occur if I run the same code via Django shell or call the method from the admin dashboard - just when the Celery task is running.
the error occurs after the following code;
from selenium import webdriver
from selenium.common.exceptions import TimeoutException, NoSuchElementException, ElementClickInterceptedException, WebDriverException
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.chrome.options import Options
...
opts = Options()
opts.headless = True
opts.add_argument("--no-sandbox")
opts.add_argument("--disable-dev-shm-usage")
opts.add_argument("--remote-debugging-port=9222")
assert opts.headless # Operating in headless mode
driver = webdriver.Chrome(options=opts)
My Django app runs on a virtual environment - /opt/python/run/venv - and chromedriver and chrome are available to it; /opt/python/run/venv/bin/chromedriver & /opt/python/run/venv/bin/google-chrome --> /opt/google/chrome/google-chrome
versions;
(venv) [ec2-user@ip-172-31-12-230 ~]$ google-chrome --version && which google-chrome
Google Chrome 83.0.4103.97
/opt/python/run/venv/bin/google-chrome
(venv) [ec2-user@ip-172-31-12-230 ~]$ chromedriver --version && which chromedriver
ChromeDriver 83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs/branch-heads/4103@{#416})
/opt/python/run/venv/bin/chromedriver
(venv) [ec2-user@ip-172-31-12-230 ~]$ pip list
Package Version
--------------------- ----------
celery 4.4.2
Django 3.0.4
django-celery-beat 2.0.0
selenium 3.141.0
I'm asking this again as my previous question was closed without an answer. It was linked to another question here by moderator and closed, however the link doesn't answer my question as I'm still getting the error. I tried making the following adjustments and validations based on the linked question;
- Google Chrome and Chromedriver are up to date
- executable path (/opt/python/run/venv/bin/chromedriver) to chromedriver was added to webdriver()
- .add_argument('--headless') was moved to be the first option set
- google-chrome in venv and /usr/bin are symlinks
- binary location option added
- tried deleting chrome and chromedriver and reinstalling (a few times)
- tried changing options to just '--no-sandbox' and '--disable-dev-shm-usage'
- I've tried putting it in a loop and adding a sleep
- Celery is not using root user or sudo
I only encounter this error when Celery worker executes tasks, I can run the same code being run by the worker in the django shell or via the admin dashboard and don't receive the error so it feels like this is a celery error and not my selenium options.
