selenium clicking accept button using XPath not working

Question

I would like to scrape data from this page: https://www.investing.com/equities/nvidia-corp-financial-summary.

There are two buttons that I'd like to click:

1. Accept button.

Checking the XPath of the button: XPath = //*[@id="onetrust-accept-btn-handler"]

Replicating the steps performed here: Clicking a button with selenium using Xpath doesn't work

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
    
wait = WebDriverWait(driver, 5)
link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id="onetrust-accept-btn-handler")))

I got the error: SyntaxError: invalid syntax

2. Annual button there is a toggle between Annual and Quarterly (default is quarterly)

XPath is //*[@id="leftColumn"]/div[9]/a[1]

wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id="leftColumn"]/div[9]/a[1]")))

also returned invalid Syntax.

---

Updated Code

from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

company = 'nvidia-corp'
driver = webdriver.Chrome(path)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")

wait = WebDriverWait(driver, 2)
accept_link= wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="onetrust-accept-btn-handler"]')))
accept_link.click()

scrollDown = "window.scrollBy(0,500);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll

driver.maximize_window()
time.sleep(10)

wait = WebDriverWait(driver, 2)

scrollDown = "window.scrollBy(0,4000);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll

try:
    close_popup_link= wait.until(EC.element_to_be_clickable((By.XPATH,'//*[@id="PromoteSignUpPopUp"]/div[2]/i')))
    close_popup_link.click()
except NoSuchElementException:
    print('No such element')
    
wait = WebDriverWait(driver, 3)
try:
    annual_link = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="leftColumn"]/div[9]/a[1]')))
    annual_link()
    # break
except NoSuchElementException:
    print('No element of that id present!')

The first accept button was successfully clicked, but clicking the Annual button returns Timeout Exception error.

---

Annual button

Answer 1

At least for me I saw we need to use another locator to access that element.
I used scrolling until I can click that element.
The following code works for me:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("--start-maximized")

s = Service('C:\webdrivers\chromedriver.exe')

driver = webdriver.Chrome(options=options, service=s)
company = 'nvidia-corp'

wait = WebDriverWait(driver, 5)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")
try:
     wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="PromoteSignUpPopUp"]/div[2]/i'))).click()
except:
    pass
while True:
    try:
        wait.until(EC.element_to_be_clickable((By.XPATH, "//div[@class='float_lang_base_1']//a[@data-ptype='Annual']"))).click()
        break
    except:
        driver.execute_script("window.scrollBy(0, arguments[0]);", 1000)

Answer 2

You need to take care of a couple of things here as follows:

• If you are supplying the xpath within double qoutes, i.e. "..." then the attribute values needs to be within single quotes, i.e. '...'
• Similarly, if you are supplying the xpath within single qoutes, i.e. '...' then the attribute values needs to be within double quotes, i.e. "..."

This take care of both the SyntaxError: invalid syntax

Effectively, the lines of code will be:

link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='onetrust-accept-btn-handler')))

and

wait.until(EC.element_to_be_clickable((By.XPATH, "//*[@id='leftColumn']/div[9]/a[1]")))

---

Solution

To click on the clickable elements you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

• Clicking on I Accept:

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
    

  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='onetrust-accept-btn-handler']"))).click()
    

• Clicking on Annual:

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[data-ptype='Annual']"))).click()
    

  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@data-ptype='Annual']"))).click()
    

• Note: You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC