How to interact with Cookie pop within #shadow-root (open) using Selenium

Question

I would like to scrap immowelt.de, but I could not get pass the cookie banner. I tried to wait for the banner to load using both sleep() and WebDriverWait, however none of them is working.

This is the code with the webdriver

from selenium import webdriver
from selenium.webdriver.common.by import By
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

chrome_driver_path = '.../chromedriver'
url = 'https://www.immowelt.de/immobilienpreise'
driver = webdriver.Chrome(executable_path=chrome_driver_path)

driver.get(url)

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="uc-center-container"]/div[2]/div/div/div/div[1]/button'))).click()

driver.close()

And this is the code with sleep

from selenium import webdriver
import time
from selenium.webdriver.common.by import By

chrome_driver_path = '.../chromedriver'
url = 'https://www.immowelt.de/immobilienpreise'
driver = webdriver.Chrome(executable_path=chrome_driver_path)

driver.get(url)
time.sleep(5)
driver.find_element(By.XPATH, '//*[@id="uc-center-container"]/div[2]/div/div/div/div[1]/button').click()

driver.close()

Answer 1

The element OK is within #shadow-root (open).

---

Solution

To click on OK you have to use shadowRoot.querySelector() and you can use the following Locator Strategy:

• Code Block:

  driver.get("https://www.immowelt.de/immobilienpreise")
  time.sleep(5)
  element = driver.execute_script("""return document.querySelector('#usercentrics-root').shadowRoot.querySelector("button[data-testid='uc-accept-all-button']")""")
  element.click()
  

---

References

You can find a couple of relevant discussions in:

• How to handle the popup "Accepting all cookies" when the element is data-testid - Using Selenium in Python
• Can't locate elments within shadow-root (open) using Python Selenium
• How to get past a cookie agreement page using Python and Selenium?