0

I'm having difficulty working with Selenium directly from Google Colab. Whenever I run the code below on my local machine I get success, however I would like to test the same application in Colab, but all the options I test I can't succeed.

import warnings
warnings.filterwarnings('ignore')
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.common.exceptions import InvalidSessionIdException
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from google.colab import drive
drive.mount('/content/gdrive', force_remount=True)

options = webdriver.ChromeOptions()
options.add_argument("--disable-blink-features=AutomationControlled")
options.add_argument('--ignore-certificate-errors-spki-list')
options.add_argument('--ignore-ssl-errors')
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option("prefs", {"profile.managed_default_content_settings.images": 2}) 
options.add_argument("--no-sandbox") 
options.add_argument("--disable-setuid-sandbox") 
options.add_argument("--disable-dev-shm-using") 
options.add_argument("--disable-extensions") 
options.add_argument("--disable-gpu") 
options.add_argument("start-maximized") 
options.add_argument("disable-infobars")
options.add_argument(r"user-data-dir=.\cookies\\test") 
options.binary_location = '/usr/bin/google-chrome'

driver = webdriver.Chrome(options=options, executable_path='/usr/bin/chromedriver') #Error occurs in this function
driver.implicitly_wait(5)

print("Current session is {}".format(driver.session_id))

I also ran the following commands:

!apt-get update
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin

After trying to run the code above, the following message appears:

**WebDriverException: Message: unknown error: no chrome binary at /usr/bin/google-chrome**

Am I doing something wrong?

PSCM
  • 85
  • 11

3 Answers3

2

I create a library to help make it simple.

!pip install kora
from kora.selenium import wd  # web driver
print(wd.session_id)  # 8be87366df11b09b552fb4ad7efbd696
korakot
  • 37,818
  • 16
  • 123
  • 144
1

Try removing this line

options.binary_location = '/usr/bin/google-chrome'

and see if it works with default binary location. Do not forget to install the driver

!apt-get update 
!apt install chromium-chromedriver
Virtuoz
  • 896
  • 1
  • 8
  • 14
  • I did what you suggested and again I had an error, however the message has changed: **WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally. (Unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)** – PSCM Sep 11 '21 at 17:06
  • It seems, some of the options cause the misconfiguration. Do you really need all these options? Usually, you can leave just the three of them: options.add_argument('--headless'), options.add_argument('--no-sandbox'), options.add_argument('--disable-dev-shm-usage'). – Virtuoz Sep 11 '21 at 17:55
0
# refer following discussion
# https://stackoverflow.com/questions/56829470/selenium-google-colab-error-chromedriver-executable-needs-to-be-in-path
# install chromium, its driver, and selenium
!apt-get update
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
!pip install selenium
# set options to be headless, ..
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
# open it, go to a website, and get results
wd = webdriver.Chrome('chromedriver',options=options)
wd.get("https://www.website.com")
print(wd.page_source)  # results
Shivendra Rajawat
  • 131
  • 2
  • 4