0

I am struggling to make selenium click on the cookie popup and access the website.

I have the following code:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import pandas as pd
from datetime import datetime
import os
import sys


web = 'https://www.thesun.co.uk/sport/football/'
path = '/Users/cc/Documents/Documents/IT/1. Python/WebCrawler/chromedriver'

driver_service = Service(executable_path=path)
driver = webdriver.Chrome(service=driver_service)
driver.get(web)
WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.XPATH,'//*.[@id="notice"]/div[4]/button[2]'))).click()

here is the webpage: enter image description here

here is the htlm button element: enter image description here

here is the vs code terminal after running: enter image description here

The webpage open but the click command does not do the work...even when defining a wait until element is clickable.

I tried to use the click command with wait to be sure the element is clickable.

the idea is to click on the "Fine By Me!" button.

Thanks for your help.

Dayta
  • 3
  • 2
  • Somehow this cookie popup doesn't surfaces when site accessed from APAC region. can you update the question with text based relevant HTML please? – undetected Selenium Jan 21 '23 at 12:04
  • @undetectedSelenium: I have added new information to my question: htlm button element details & vs code terminal. Thanks. – Dayta Jan 21 '23 at 19:05

2 Answers2

0

You need to switch to the iframe and then click on the button. Also there is a browser pop-up for allowing/blocking notifications. You need to handle that in the driver setup.

Driver Setup can be done with params

chromeOpts = Options()
chromeOpts.add_argument('start-maximized')
chromeOpts.add_argument("--disable-notifications")
chromeOpts.add_argument("--disable-infobars")
chromeOpts.add_argument("--disable-extensions")

The code to switch to iframe and then clikcing on button is -

iframe = "//iframe[@title='SP Consent Message']"

ifr = driver.find_element(By.XPATH, iframe)
driver.switch_to.frame(ifr)

WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH,'//*[@id="notice"]/div[4]/button[2]'))).click()

Here - enter image description here

Manish
  • 484
  • 3
  • 12
  • Hi Manish, thank you. Not sure to see the difference btw your suggestion and the code I am proposing other than the wait time. I tried with 10 sec but no change. Or is there another difference I missed?thx – Dayta Jan 21 '23 at 19:10
  • There should be no dot after the * in the xpath – Manish Jan 21 '23 at 23:13
  • missed that one ;) thanks. It does not work though. – Dayta Jan 22 '23 at 14:43
  • increase the wait and remove the dot. It worked for me. Should work for you as well – Manish Jan 22 '23 at 20:56
  • if it worked for you then something else is preventing my code from working because even with the dot out and the wait at 30, it does nothing to the cookie popup. Any thought? (e.g. selenium version issue). Thx. – Dayta Jan 23 '23 at 18:09
  • @Dayta please try again with the updated changes regarding iframe switch. It will work now :) – Manish Jan 23 '23 at 21:37
0

To click on the element Fine By Me! you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[title='Fine By Me!'][aria-label='Fine By Me!']"))).click()

  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@title='Fine By Me!' and @aria-label='Fine By Me!']"))).click()

  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • @undetecded Selenium: thanks for you reply. I tried the solutions you provided but both are ineffective with this site. However, i used them on another website that uses an id element and it works. So, without id element it seems not working. There is a notification popup asking for location permission coming on top: not sure if it interferes with your code. – Dayta Jan 22 '23 at 14:39
  • @Dayta Generally these issues are quite common specifically when you don't supply the text based HTML. Apart from that the element can be within an _iframe_ ot _shadow-root_. So humongous possibilities for the proposed solution to fail. – undetected Selenium Jan 22 '23 at 14:55
  • @undetecded Selenium:Thanks. I thought I have updated my question with the text based html ( in my question => "here is the htlm button element" link). – Dayta Jan 22 '23 at 16:39