1

For context I am working on a way to automate getting specific bits of information from my company's bookings engine. I'm using selenium to log into the bookings engine and navigate to the relevant pages to get the information I want. I want to be able to do this without opening an actual visible browser window.

I have been working with a visible browser for the purposes of helping me actually see what the code was doing when I run it making it easier to fix bugs. However when I tried to run it with a headless browser I suddenly got an element not interactable exception. Here's the relevant parts of my code.

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options

option = webdriver.ChromeOptions()
option.add_argument('headless')

driver = webdriver.Chrome(ChromeDriverManager().install(), options=option)

My code is identical to what it was before. The only thing I have changed is the addition of the 'headless' argument to the options. Anyone know why this would cause some elements to not be interactable and how to fix it?

Michael Mintz
  • 9,007
  • 6
  • 31
  • 48
thisisminenow
  • 13
  • 2

1 Answers1

0

The solution for this involves using https://github.com/ponty/PyVirtualDisplay in order to simulate a headed display in a headless environment such as Linux. (This can help you get around issues that might only appear when loading a website with a headless browser.) The virtual display uses Xvfb, and there's actually a very popular Stack Overflow solution here about it: https://stackoverflow.com/a/6300672

Here's a summary of the previous solution for your situation:

To install:

pip install pyvirtualdisplay

And here's how to use it in your Python code:

from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
# display is active / Run your selenium code
# ----------
display.stop()
# display is stopped

After putting it all together with your code, it will look something like this:

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.options import Options
from pyvirtualdisplay import Display

display = Display(visible=0, size=(800, 600))
display.start()
# ...
option = webdriver.ChromeOptions()
driver = webdriver.Chrome(ChromeDriverManager().install(), options=option)
# ...
display.stop()

(Make modifications as necessary, such as the screen resolution, etc.)

There are some Selenium Python frameworks that already have the virtual display built-in (such as SeleniumBase, which activates the virtual display via an added pytest command-line arg: --xvfb.) Each framework that has the feature built-in may do things a bit differently.

Just be sure NOT to run Chrome in headless mode if you use this solution. The virtual display should allow you to run a regular headed Chrome in the headless environment.

Michael Mintz
  • 9,007
  • 6
  • 31
  • 48