My aim is to automate the downloading of 1-minute tick data (csv) from Fidelity.
The issue is that the webdriver doesn't open the dropdown that displays the different interval options.
The ways I've determined that the button is not being clicked are that, as written below, the python code prints only the 'ElementNotInteractableException' statement, and I don't see the dropdown open when the code is run.
What I've tried so far is both the xpath and the selector path for the dropdown button element, titled 'DAILY'. I've also attempted using elements surrounding the dropdown button in the HTML code, such as a little triangle next to 'DAILY'. I also tried using Select but got an error since element was a div.
Of relevance perhaps, I am also unable to click other buttons in a container containing the interval dropdown, such as the buttons which change the range of the prices displayed. And I've managed to open up other dropdowns on the screen, namely the dropdown that allows one to download the csv. Lastly, the 'unclickable' buttons are near the bottom of the screen.
All suggestions welcome. If there are code details or html snippets to include, let me know.
The website (Fidelity account needed to access): https://screener.fidelity.com/ftgw/etf/gotoCL/snapshot/advancedChart.jhtml?symbols=KO
The python code:
from selenium import webdriver as webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By as By
import selenium.common.exceptions as exceptions
from selenium.webdriver.support import (
expected_conditions as EC)
import time
class Scraper:
def __init__(self):
self.driver = webdriver.Safari()
def scrapes(self):
try:
self.signIn()
time.sleep(3)
self.getCSV('KO')
except exceptions.TimeoutException:
print('TimeoutException')
except exceptions.ElementNotInteractableException:
print('ElementNotInteractableException')
self.driver.close()
def signIn(self):
driver = self.driver
driver.get('https://www.fidelity.com/')
#enter username
ecTuple = (By.CSS_SELECTOR, 'input#userId-input')
elPresent = EC.element_to_be_clickable(ecTuple)
WebDriverWait(driver, 30).until(elPresent).send_keys(
"USERNAME")
#enter password
driver.find_element_by_css_selector(
'input#password').send_keys("PASSWORD")
#click login button
driver.find_element_by_css_selector(
'button#fs-login-button').click()
def getCSV(self, tickerStr):
driver = self.driver
stockURL = (
'https://screener.fidelity.com/' +
'ftgw/etf/gotoCL/snapshot/advancedChart' +
'.jhtml?symbols=' + tickerStr
)
driver.get(stockURL)
# open intervals dropdown
drpDnXP = '//*[@id="selected_frequency"]'
ecTuple = (By.XPATH, drpDnXP)
elPresent = EC.element_to_be_clickable(ecTuple)
WebDriverWait(driver, 30).until(elPresent).click()
# choose one minute intervals
minute1XP = '//*[@id="frequency_0"]'
driver.find_element_by_xpath(minute1XP).click()
print(1)
Scraper().scrapes()
The html of the dropdown button element in question:
<div id="selected_frequency" class="ng-binding ng-scope">DAILY</div>
Some surrounding html code:
<div class="selected-item-container">
<div class="selected-item ng-binding">
<div id="selected_frequency" class="ng-binding ng-scope">DAILY</div>
</div>
<span class="triangle"></span>
</div>