I'm using Selenium in JupyterLab on IBM's CognitiveClass Labs, and although the package is easy enough to install there with pip (actually I think it already came pre-loaded), it can't find the drivers it needs in PATH:
from selenium import webdriver
browser = webdriver.Firefox()
[Out] FileNotFoundError: [Errno 2] No such file or directory: 'geckodriver': 'geckodriver'
[Out] WebDriverException: Message: 'geckodriver' executable needs to be in PATH.
It's the same for all other browsers too, just replace 'geckodriver' with the Chrome/IE substitute.
Anyway, this would probably be straightforward on a real python installation but I prefer to use it in JupyterLab if I can. I downloaded the driver .exe files and placed them in my project's directory, which the lab sees as /resources/myproj. Then I added this to the PATH that JupyterLab was already using, and specified the location of the executable for Selenium:
%env PATH=/home/jupyterlab/conda/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/resources/myproj
driver = os.path.normpath(os.path.join(os.getcwd(), 'geckodriver.exe'))
browser = webdriver.Firefox(executable_path=driver)
[Out] PermissionError: [Errno 13] Permission denied: '/resources/myproj/geckodriver.exe'
[Out] WebDriverException: Message: 'geckodriver.exe' executable may have wrong permissions.
Can I (or must I) modify permissions on these files in JupyterLab in order for Selenium to access them? Or is there another way to mimic a browser in Jupyter?
Edit: Corey Goldberg is right, this is a Linux environment not a Windows one and I was able to chmod the linux driver to get over that particular issue. But Selenium is still stopping me.
[Out] SessionNotCreatedException: Message: Unable to find a matching set of capabilities
The questions I have (which make this a very different issue from the one DebanjanB cited) are JupyterLab-specific:
- Is it dependent on my local Windows machine's Firefox installation? It has a much older version (52.5.2), so I tried the corresponding version of the gecko driver (17.0) in addition to the latest version. But I doubt that, because I'm not pointing it to Firefox's executable on my local drive.
- Do I need the 32-bit or 64-bit Linux driver? I tried both anyway, both return the same error above. But I still don't know because from what I understand my code is running on IBM's computers, not my own.
Edit2: Resolution
This issue may be specific to my work environment. I installed the latest Firefox to my lab's folder using JupyterLab's terminal
$ cd /tmp
$ wget 'http://download.mozilla.org/?product=firefox-latest-ssl&os=linux64&lang=en-US' -O firefox-67.0.4.tar.bz2
$ tar jxvf firefox-67.0.4.tar.bz2 -C /resources/myproj/
then set marionette capability to False, created an explicit Firefox binary so I could look at the log (ultimately it didn't write anything, don't know why)
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
cap = DesiredCapabilities().FIREFOX
cap["marionette"] = False
driver = os.path.normpath(os.path.join(os.getcwd(), 'geckodriver'))
binary = os.path.normpath(os.path.join(os.getcwd(), 'firefox', 'firefox'))
ff_binary = webdriver.firefox.firefox_binary.FirefoxBinary(firefox_path=binary, log_file='ff_log.log')
browser = webdriver.Firefox(firefox_binary=ff_binary, capabilities=cap, executable_path=driver)
browser.get('http://google.com/')
In the end it seems like something on my side is stopping it, and it's gone beyond the scope of my original question.
[Out] WebDriverException: Message: The browser appears to have exited before we could connect. If you specified a log_file in the FirefoxBinary constructor, check it for details.
