I have a Flask app that runs a Python script using Selenium, and it receives POST requests with a url that Selenium is supposed to open. The Python script is hosted on a VM in Google Cloud, and when I run the code using Flask it works just fine, and the POST requests sent to the server get parsed just fine. Everything works as intended and as supposed to.
I've been trying to deploy it with Nginx, Gunicorn, and systemd to have 24/7 uptime, but whenever a POST request gets passed to my server, I get the following error in my logs.log file:
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed.
(chrome not reachable)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
I've looked at every other thread on StackOverflow about this issue and all of them seem to stem from chromedriver configuration issues or non-compatible versions of Chrome and chromeDriver, but since it runs completely fine on Flask itself but only breaks once I am routing it through Nginx and Gunicorn, I highly doubt this is the case. I have checked my versions of chrome and chromeDriver, and they seem to be fine (and, in general, I'm fairly certain all of my paths are right elsewhere in my code as well):
chromedriver --version
ChromeDriver 115.0.5790.170
google-chrome --version
Google Chrome 115.0.5790.170
selenium
Version: 4.11.2
So, for reference, here is the configuration I have in my Python file:
from flask import Flask, request, jsonify
from flask_cors import CORS
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
import time
import logging
logging.basicConfig(filename='/home/username/logs.log', level=logging.INFO)
app = Flask(__name__)
CORS(app)
def open_url_in_selenium(url):
service = Service('/usr/local/bin/chromedriver')
service.start()
options = webdriver.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--disable-extensions')
options.add_argument('--remote-debugging-port=9222')
options.add_argument('--log-level=DEBUG')
options.add_argument('--enable-logging')
options.add_argument('--disable-software-rasterizer')
options.binary_location = '/usr/bin/google-chrome'
options.add_argument('--log-path=' + log_file_path)
driver = webdriver.Chrome(options=options)
driver.get(url)
return driver
and here is the systemd Service Unit, if relevant:
[Unit]
Description=XYZ thing
After=network.target
[Service]
User=username
Group=username
WorkingDirectory=/home/username
ExecStart=/usr/bin/gunicorn --workers 1 --bind X.X.X.X:XXXX server:app
Restart=always
[Install]
WantedBy=multi-user.target
AND one last thing to note: for some reason, even when I run my code on Flask, I need to use sudo python3 filename.py to actually get the code to run without the permission being denied. Could this be a permissions issue? When I run systemd I use sudo as well, so I thought it would be fine, but maybe not? Here are the commands I have been using:
sudo systemctl start service-unit-name
sudo systemctl enable service-unit-name
sudo systemctl stop service-unit-name
sudo systemctl restart service-unit-name
If this is the problem, would someone be able to explain how to configure the permissions properly? And would anyone happen to know what is going on? I have been trying to figure this out for hours! Thanks so much.
