selenium.common.exceptions.ElementClickInterceptedException error clicking on a toggle button on yahoo finance income statement data

Question

I am trying to webscrape Income Statement data from https://finance.yahoo.com/quote/AAPL/financials?p=AAPL

I am trying to simulate a click on Operating Expense to expand the row to see the Research and Development values like such:

This is my code so far:

def clickOperatingExpense(url):
        options = Options()
        options.add_argument('--headless')
        driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
        driver.get(url)
        # This is where I encounter a selenium.common.exceptions.ElementClickInterceptedException
        WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="Col1-1-Financials-Proxy"]/section/div[4]/div[1]/div[1]/div[2]/div[4]/div[1]/div[1]/div[1]/button'))).click()

Just for some more information, the full error is:

selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button aria-label="Operating Expense" class="P(0) M(0) Va(m) Bd(0) Fz(s) Mend(2px) tgglBtn">...</button> is not clickable at point (39, 592). Other element would receive the click: <div tabindex="1" class="lightbox-wrapper Ta(c) Pos(f) T(0) Start(0) H(100%) W(100%) Bgc($modalBackground) Ovy(a) Z(50) Op(1)">...</div>

The issue looks like a div tag would receive the click but I'm not sure why. How can I make the tag receive the click instead?

undetected Selenium · Accepted Answer · 2023-03-06T21:05:40.213

1

To click and expand the element with text as Operating Expense you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

Using CSS_SELECTOR:

driver.get("https://finance.yahoo.com/quote/AAPL/financials?p=AAPL")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='Close']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[title='Operating Expense'] > button[aria-label='Operating Expense']"))).click()

Using XPATH:

driver.get("https://finance.yahoo.com/quote/AAPL/financials?p=AAPL")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@aria-label='Close']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@title='Operating Expense']/button[@aria-label='Operating Expense']"))).click()

Browser Snapshot:

OperatingExpence

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

edited Mar 06 '23 at 21:05

answered Mar 06 '23 at 20:58

undetected Selenium

183,867
41
278
352

This solution worked. Quick question: Why does this line ```WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@aria-label='Close']"))).click()``` fix my issue? What is this line of code actually clicking? – AJ Goudel Mar 06 '23 at 21:21
`WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@aria-label='Close']"))).click()` clicks and closes the popup which obstructs the click on the desired element. – undetected Selenium Mar 06 '23 at 21:23
The script is able to run successfully most of the time but I sometimes encounter a similar error but on a different tag and it seems to be a threading issue: ```Other element would receive the click: ... ``` This time the tag is intercepting the click instead of a
tag
– AJ Goudel Mar 09 '23 at 19:38
@AJGoudel Sounds to be a different issue. Feel free to raise a new question as per your new requirements. – undetected Selenium Mar 09 '23 at 19:51

score 0 · Answer 2 · answered Mar 06 '23 at 20:48

0

Try using Java script to perform click() as below:

driver.execute_script("arguments[0].click();", element)

Reference: https://stackoverflow.com/a/75645682/7598774

answered Mar 06 '23 at 20:48

Shawn

4,064
2
11
23

selenium.common.exceptions.ElementClickInterceptedException error clicking on a toggle button on yahoo finance income statement data

2 Answers2