Navigating a website headless in Chrome using Selenium Python

Question

I'm trying to figure out how to manipulate a webpage in headless mode using Selenium Python but I keep getting an error message. When I run the below code visible it runs without an issue.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time

options = Options()
options.set_headless(headless=True)
options.add_argument('window-size=1200x600')
driver = webdriver.Chrome(options=options, executable_path=r'F:\\Python\\chromedriver.exe')
driver.get("https://www.google.com/")

element = driver.find_element_by_name('q')

element.send_keys('test')

test1 = driver.find_element_by_name('btnK')
test1.click()

print ("Test Completed")
driver.quit()

Error message:

Traceback (most recent call last):
  File "F:\Python\URLtest.py", line 154, in <module>
    test1.click()
  File "C:\Users\User1\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium-3.14.0-py3.6.egg\selenium\webdriver\remote\webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "C:\Users\User1\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium-3.14.0-py3.6.egg\selenium\webdriver\remote\webelement.py", line 628, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\User1\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium-3.14.0-py3.6.egg\selenium\webdriver\remote\webdriver.py", line 320, in execute
    self.error_handler.check_response(response)
  File "C:\Users\User1\AppData\Local\Programs\Python\Python36-32\lib\site-packages\selenium-3.14.0-py3.6.egg\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotVisibleException: Message: element not visible
  (Session info: headless chrome=68.0.3440.106)
  (Driver info: chromedriver=2.41.578737 (49da6702b16031c40d63e5618de03a32ff6c197e),platform=Windows NT 6.1.7601 SP1 x86_64)

score 0 · Answer 1 · answered Sep 17 '18 at 15:26

You can use a try except to prevent this error from happening.

in try : section you put your click command.

in the except ElementNotVisibleException : you can set the visibility to visible with a execute_script then click.

This should looks like

driver.execute_script("arguments[0].style.visibility = 'visible';",Element)
Element.click()

score 0 · Accepted Answer · answered Sep 17 '18 at 15:44

0

This can happen if the element in question is not in the view port of the browser.

This can be fixed by executing some script or using an ActionChain to scroll to the element.

From this SO question:

driver.execute_script("arguments[0].scrollIntoView();", element)

answered Sep 17 '18 at 15:44

Rodrigo Ce Moretto

198
2
8

This was helpful thanks! – Eric Sep 17 '18 at 17:08

Navigating a website headless in Chrome using Selenium Python

2 Answers2