0

My aim is to automate the downloading of 1-minute tick data (csv) from Fidelity.

The issue is that the webdriver doesn't open the dropdown that displays the different interval options.

The ways I've determined that the button is not being clicked are that, as written below, the python code prints only the 'ElementNotInteractableException' statement, and I don't see the dropdown open when the code is run.

What I've tried so far is both the xpath and the selector path for the dropdown button element, titled 'DAILY'. I've also attempted using elements surrounding the dropdown button in the HTML code, such as a little triangle next to 'DAILY'. I also tried using Select but got an error since element was a div.

Of relevance perhaps, I am also unable to click other buttons in a container containing the interval dropdown, such as the buttons which change the range of the prices displayed. And I've managed to open up other dropdowns on the screen, namely the dropdown that allows one to download the csv. Lastly, the 'unclickable' buttons are near the bottom of the screen.

All suggestions welcome. If there are code details or html snippets to include, let me know.

The website (Fidelity account needed to access): https://screener.fidelity.com/ftgw/etf/gotoCL/snapshot/advancedChart.jhtml?symbols=KO

The python code:

from selenium import webdriver as webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By as By
import selenium.common.exceptions as exceptions
from selenium.webdriver.support import (
    expected_conditions as EC)
import time

class Scraper:

    def __init__(self):

        self.driver = webdriver.Safari()

    def scrapes(self):

        try:

            self.signIn()
            time.sleep(3)
            self.getCSV('KO')

        except exceptions.TimeoutException:

            print('TimeoutException')

        except exceptions.ElementNotInteractableException:

            print('ElementNotInteractableException')

        self.driver.close()

    def signIn(self):

        driver = self.driver
        driver.get('https://www.fidelity.com/')

        #enter username
        ecTuple = (By.CSS_SELECTOR, 'input#userId-input')
        elPresent = EC.element_to_be_clickable(ecTuple)
        WebDriverWait(driver, 30).until(elPresent).send_keys(
            "USERNAME")

        #enter password
        driver.find_element_by_css_selector(
            'input#password').send_keys("PASSWORD")

        #click login button
        driver.find_element_by_css_selector(
            'button#fs-login-button').click()

    def getCSV(self, tickerStr):

        driver = self.driver
        stockURL = (
            'https://screener.fidelity.com/' +
            'ftgw/etf/gotoCL/snapshot/advancedChart' +
            '.jhtml?symbols=' + tickerStr
            )
        driver.get(stockURL)

        # open intervals dropdown
        drpDnXP = '//*[@id="selected_frequency"]'
        ecTuple = (By.XPATH, drpDnXP)
        elPresent = EC.element_to_be_clickable(ecTuple)
        WebDriverWait(driver, 30).until(elPresent).click()

        # choose one minute intervals
        minute1XP = '//*[@id="frequency_0"]'
        driver.find_element_by_xpath(minute1XP).click()

        print(1)

Scraper().scrapes()

The html of the dropdown button element in question:

<div id="selected_frequency" class="ng-binding ng-scope">DAILY</div>

Some surrounding html code:

<div class="selected-item-container">
        <div class="selected-item ng-binding">
                <div id="selected_frequency" class="ng-binding ng-scope">DAILY</div>
        </div>
        <span class="triangle"></span>
</div>
  • The problem element being near the edge of the page sounds suspicious... have you tried scrolling the page to make it more central? These maybe useful: https://stackoverflow.com/questions/3401343/scroll-element-into-view-with-selenium , https://stackoverflow.com/questions/48731466/scroll-up-and-down-to-get-element-into-view-with-selenium-python/53771434#53771434 – timday May 25 '20 at 20:43
  • @timday I've since implemented this but did not quite work, resulted in MoveTargetOutOfBoundsException error. Thanks for suggestion though. I now believe the problem is that the driver is not choosing the right element. – Alfonso May 26 '20 at 00:39

0 Answers0