Can't find element belonging to 'Accept All' button

Question

I recently started learning Selenium and webscraping in Python. I'm trying to find and click the 'Accept All' button on the pop-up (image of the pop-up can be found below) when entering the following site: https://www.sherdog.com, using Chrome. It takes around 5 seconds for the pop-up to load. I have tried different things and have red what I could find on stackoverflow describing similar problems. To no avail. I always get the NoSuchElementException (or NoAlertPresentException).

I have tried the following things:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver.get('https://www.sherdog.com')
driver.find_element(By.CLASS_NAME, 'Button__StyledButton-a1qza5-0 incZp')
driver.switch_to.alert
try:
    element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, 'Button__StyledButton-a1qza5-0 incZp')))
except:
    print("An exception occurred")

I also thought I might have to switch frame using driver.switchTo().frame(driver.findElement(By.id("rufous-sandbox"))), but am honestly unsure which frame to select. When looking through the HTML code (which I just started learning) I see some references to JavaScript (of which I have zero knowledge). Maybe that is causing me trouble?

If anybody could provide some insight, or point me in the right direction, would be greatly appreciated.

Answer 1

1. There are 2 objects you need to close there and they are not alerts. So driver.switch_to.alert is not relevant here.
2. Always try using stable unique locators. Class names like incZp may often be dynamic and not reliable.
3. Button__StyledButton-a1qza5-0 incZp are actually 2 class names, so you have to use CSS_SELECTOR or XPATH to work with them.
4. It is always preferred to wait for element visibility, not just existence when you going to click on that element.

This should work:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver.get('https://www.sherdog.com')
wait = WebDriverWait(driver, 20)

wait.until(EC.visibility_of_element_located((By.XPATH, "//button[contains(text(),'Continue')]"))).click()
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div#cookieNotice a.cnaccept"))).click()

Answer 2

To click on the element Accept Cookies you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

• Using CSS_SELECTOR:

  driver.get("https://www.sherdog.com")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div>a.cnaccept"))).click()
  

• Using XPATH:

  driver.get("https://www.sherdog.com")
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='cnaccept']"))).click()
  

• Note: You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

Answer 3

This is how you click that element:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys

chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('disable-notifications')
chrome_options.add_argument("window-size=1920,1080")

webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)
actions = ActionChains(browser)

url = 'https://www.sherdog.com'
browser.get(url) 
elem = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@title='Scroll to the bottom of the text below to enable this button']")))

elem.click()
print('clicked')

Bear in mind that, if window is not sufficiently large, that text will need to be scrolled (and button will not be clickable). Default headless window size is quite small, so make sure your window is sufficiently large. Selenium docs: https://www.selenium.dev/documentation/