2

I am quite new to selenium and python. I am trying to navigate through a webpage (https://www.heise.de/download/) but can't get passed the cookiewall (cookie agreement page). It seems that the webdriver can't find the button to click on.

The HTML code:

<button type="button" backgroundcolor="[object Object]" data-testid="uc-accept-all-button" fullwidth="true" class="sc-bdnylx fmRkNf">Alles akzeptieren</button>

Code trials:

from selenium import webdriver
from time import sleep
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Using Firefox to access web
driver = webdriver.Firefox(executable_path=r'C:\Users\aphro\Anaconda3\geckodriver.exe')

# Open the website
driver.get('https://www.heise.de/download/')
sleep(5)
if driver.find_element_by_xpath('//*[@id="usercentrics-root"]'):
    print("found popup window")

# find button class
wait = WebDriverWait(driver, 10)
element = wait.until(EC.element_to_be_clickable((By.XPATH, '//div[@class="sc-bdnylx fmRkNf"]')))
element.click()

Error message:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/aphro/PycharmProjects/SoftwareProject/SoftwareProject/webControl.py", line 18, in <module>
    x = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div[data-testid='uc-accept-all-button'][role='button']"))).click()
  File "C:\Users\aphro\AppData\Roaming\Python\Python38\site-packages\selenium\webdriver\support\wait.py", line 80, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

I've tried different methods including CSS Selector and XPATH. I use Python 3.8 and Selenium 3.141.

Can anyone point me in the right direction on how to bypass this cookiewall (artificially 'click' the button) and go to the actual webpage?

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
aphrodite
  • 31
  • 4

1 Answers1

1

The element Alles akzeptieren is within #shadow-root (open).

Alles

Solution

To click on Alles akzeptieren you have to use shadowRoot.querySelector() and you can use the following Locator Strategy:

  • Code Block:

    driver.get("https://www.heise.de/download/")
time.sleep(5)
element = driver.execute_script("""return document.querySelector('#usercentrics-root').shadowRoot.querySelector('footer div div div button[data-testid="uc-accept-all-button"]')""")
element.click()

  • Browser Snapshot:

heise

References

You can find a couple of relevant discussions in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352