0

I'm trying to run Selenium inside the context of a Flask application running on Apache as a WSGI application inside a venv.

(I strongly suspect the problem is permission-related but can't figure out the solution.)

Flask code

from selenium import webdriver
from selenium.webdriver import FirefoxOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options

def create_session():

  chrome_options = Options()
  chrome_options.add_argument('--no-sandbox')
  chrome_options.add_argument('--headless')
  chrome_options.add_argument('--disable-dev-shm-usage')
  chrome_options.binary_location="/snap/bin/chromium"
  chrome_options.add_argument("--log-path='/path/to/app/geckodriver.log'")
  service = Service('/usr/bin/chromedriver')
  browser=webdriver.Chrome(service=service, options=chrome_options)

  browser.get('https://someaddress.com/login')

Route

@app.route('/test')
def test():
  r = create_session()

Error (in apache2/error.log)

[...] [wsgi:error] [...] [client ...]   File "/path/to/app/selenium-stuff.py", line 34, in create_session
[...] [wsgi:error] [...] [client ...]     browser = webdriver.Firefox(service=service, options=opts)
[...] [wsgi:error] [...] [client ...]   File "/path/to/app/venv/lib/python3.10/site-packages/selenium/webdriver/firefox/webdriver.py", line 61, in __init__
[...] [wsgi:error] [...] [client ...]     self.service.start()
[...] [wsgi:error] [...] [client ...]   File "/path/to/app/venv/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 97, in start
[...] [wsgi:error] [...] [client ...]     self.assert_process_still_running()
[...] [wsgi:error] [...] [client ...]   File "/path/to/app/venv/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 110, in assert_process_still_running
[...] [wsgi:error] [...] [client ...]     raise WebDriverException(f"Service {self._path} unexpectedly exited. Status code was: {return_code}")
[...] [wsgi:error] [...] [client ...] selenium.common.exceptions.WebDriverException: Message: Service /snap/bin/geckodriver unexpectedly exited. Status code was: 1

I've tried --

  • ensuring headless
  • switching to chromium (chrome-driver)

geckodriver.log exists, has had chmod +w, but doesn't hold much in the way of logging. /snap/bin/geckodriver similarly exists and should be globally executable.

The function works perfectly well when (a) called directly through python3 on the command line, or (b) called through the Flask development webserver, but not when used in Apache/WSGI. Hence why I suspect a permissions issue...

edit... Tried running /usr/bin/chromedriver as normal, non-privileged (i.e. non-root) user -- works fine. Tried running sudo -u www-data /usr/bin/chromedriver and got the following:

/user.slice/user-1000.slice/session-2.scope is not a snap cgroup

so it looks like chromedriver can't be run as user www-data. I tried disabling (temporarily!) apparmor.service, but that made no difference.

Solution?

I eventually gave up with chromium/chromedriver and went back to firefox.

It's not a tidy solution, but ... following (post #13 here) suggested reinstalling firefox with developer mode enabled, which circumvents some of the sandboxing permission issues... which appears to have worked.

jrs44
  • 1
  • 2
  • Did you try using `Selenium` plain sight? Did you try to use `Selenium` with Flask in it? Did you try to use `Selenium` with Flask **AND** hosted on an apache? I manage to make it work in the Flask context. But I didn't try with Apache. I feel like this problem is linked to how `Selenium` is installed on your system – RaiZy_Style Jul 10 '23 at 12:35
  • Works correctly when called directly through python3 on the command line, and when the Flask route is accessed through the Flask development webserver, but **not** when accessed through Apache2/WSGI. – jrs44 Jul 10 '23 at 13:22
  • Hum, so it's more about how Apache interact with Flask that itself interact with Selenium. Perhaps Apache doesn't have the good permission to call selenium driver ? I don't really know how apache interact with Python and flask. – RaiZy_Style Jul 10 '23 at 13:28
  • Can you try this [solution](https://stackoverflow.com/questions/61174870/how-to-install-selenium-python-on-a-apache-web-server) ? Where they said that you can have the driver installed anywhere. Might worth the shot to move ChromeDriver to the folder where `Apache` is hosted (`/var/www/html/my-app`) so www-data can have access to it by default. – RaiZy_Style Jul 10 '23 at 13:43
  • I did try that -- or at least, something very simlar, trying options.binary_location="/snap/bin/chromium" -- no joy. Think it's a permission issue with the Apache user (www-data) being unable to start a chromium / chrome-driver instance. – jrs44 Jul 10 '23 at 13:51

1 Answers1

0

Solution?

I eventually gave up with chromium/chromedriver and went back to firefox.

It's not a tidy solution, but ... following (post #13 here) suggested reinstalling firefox with developer mode enabled, which circumvents some of the sandboxing permission issues... which appears to have worked.

jrs44
  • 1
  • 2