How to click this button using selenium webdriver?

Question

I am trying to scrape the data table from nasdaq: https://www.nasdaq.com/symbol/msft/interactive-chart?timeframe=5d

What I do is using python and selenium webdriver to click the table button(on top of the chart, with a little table logo) and then scrape.

submit = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#dataTableBtn')))
submit.click()

But it does not work.

Button html here:

<div id="dataTableBtn" class="btn hideSmallIR stx-collapsible" onclick="dataTableLoader()"><span>Data Table</span></div>

EC and By

from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Its inside ``, you need to switch to it first. – Guy Aug 07 '19 at 08:52 — Guy, Aug 07 '19 at 08:52

undetected Selenium · Accepted Answer · 2019-08-07T09:44:35.020

The chart and the associated elements are within an <iframe> so you have to:

Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.

You can use either of the following Locator Strategies:

Using CSS_SELECTOR:

driver.get("https://www.nasdaq.com/symbol/msft/interactive-chart?timeframe=5d")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src*='edgar-chartiq']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.hideSmallIR#dataTableBtn>span"))).click()

Using XPATH:

driver.get("https://www.nasdaq.com/symbol/msft/interactive-chart?timeframe=5d")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(@src, 'edgar-chartiq')]")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='btn hideSmallIR stx-collapsible' and @id='dataTableBtn']/span[text()='Data Table']"))).click()

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Browser Snapshot:

Here you can find a relevant discussion on Ways to deal with #document under iframe

Why don't you use `'#chartholder > iframe'` as the CSS SELECTOR? — Moshe Slavin, Aug 07 '19 at 09:12
@MosheSlavin From a generic approach, the search for a direct `iframe` tag will be much more faster. However `'#chartholder > iframe'` would also work. — undetected Selenium, Aug 07 '19 at 09:15
Yes it works! I am new to web scraping so I am not familiar with iframe. Thanks so much for your help! — frifin, Aug 07 '19 at 09:23
@DebanjanB Your locator to find the IFRAME vs MosheSlavin's will be negligible in speed difference. Where are you getting, `the search for a direct iframe tag will be much more faster`? — JeffC, Aug 07 '19 at 14:13

score 1 · Answer 2 · answered Aug 07 '19 at 08:56

The table is in an iframe, so just switch to it!

frame = driver.find_element_by_css_selector('#chartholder > iframe')
driver.switch_to.frame(frame)

You can use WebDriverWait with EC like this:

frame = driver.find_element_by_css_selector('#chartholder > iframe')
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it(frame))

score -1 · Answer 3 · answered Aug 07 '19 at 09:02

The button you're looking for belongs to an iframe so you will have to switch to it prior to attempting to find the button.

Both iframe and the button can be located using XPath contains() function

driver.switch_to.frame(driver.find_element_by_xpath("//iframe[contains(@src,'edgar-chartiq')]"))

Once you're in the iframe you should be able to locate and click the button:

driver.find_element_by_xpath("//*[contains(text(),'Data Table')]").click()

Why use `XPath` and `contains()` when you can use CSS? – Moshe Slavin Aug 07 '19 at 09:05 — Moshe Slavin, Aug 07 '19 at 09:05

How to click this button using selenium webdriver?

3 Answers3