How to select an option from the dropdown menu using Selenium and Python

Question

I am trying to get the fares for all combinations of stations in Metro-North using Selenium in Python. I wanted to go to their fare page, put station names into the select fields, click on the fare button, and then copy the needed values to the dataframe.

I tried all possible options to select stations from the dropdown menu but nothing works as I get the error: ElementNotInteractableException.

The code I tried:

driver = webdriver.Safari()
driver.get('http://as0.mta.info/mnr/schedules/sched_form.cfm')

select = Select(driver.find_element_by_id('Vorig_station'))
print([o.text for o in select.options])
time.sleep(3)
select.select_by_visible_text('ANSONIA')
element = driver.find_element_by_xpath('//*[@id="frmindex"]/table[2]/tbody/tr[6]/td/input[2]')
element.click()
driver.close()

print command shows the options but I cannot choose and go to the next fares page.

Help please!

Answer 1

To get the fares for all combinations of stations in Metro-North using Selenium and python selecting the option ANSONIA from the From Staion drop-down-menu you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following Locator Strategies:

• Using CSS_SELECTOR:

  driver.get("http://as0.mta.info/mnr/schedules/sched_form.cfm")
  WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select#Vorig_station"))).click()
  select = Select(WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select#Vorig_station"))))
  print([o.text for o in select.options])
  select.select_by_visible_text('ANSONIA')
  WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[value='fares']"))).click()
  

• Using XPATH:

  driver.get("http://as0.mta.info/mnr/schedules/sched_form.cfm")
  WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//select[@id='Vorig_station']"))).click()
  select = Select(WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//select[@id='Vorig_station']"))))
  print([o.text for o in select.options])
  select.select_by_visible_text('ANSONIA')
  WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//input[@value='fares']"))).click()
  

• Console Output:

  ['ANSONIA', 'APPALACHIAN TRAIL', 'ARDSLEY-ON-HUDSON', 'BEACON', 'BEACON FALLS', 'BEDFORD HILLS', 'BETHEL', 'BOTANICAL GARDEN', 'BRANCHVILLE', 'BREAKNECK RIDGE', 'BREWSTER', 'BRIDGEPORT', 'BRONXVILLE', 'CANNONDALE', 'CHAPPAQUA', 'COLD SPRING', 'CORTLANDT', 'COS COB', 'CRESTWOOD', 'CROTON FALLS', 'CROTON-HARMON', 'DANBURY', 'DARIEN', 'DERBY', 'DOBBS FERRY', 'DOVER PLAINS', 'EAST NORWALK', 'FAIRFIELD', 'FAIRFIELD METRO', 'FLEETWOOD', 'FORDHAM', 'GARRISON', 'GLENBROOK', 'GLENWOOD', 'GOLDENS BRIDGE', 'GRAND CENTRAL', "GREEN'S FARMS", 'GREENWICH', 'GREYSTONE', 'HARLEM - 125TH ST.', 'HARLEM VALLEY-WINGDALE', 'HARRISON', 'HARTSDALE', 'HASTINGS-ON-HUDSON', 'HAWTHORNE', 'IRVINGTON', 'KATONAH', 'LARCHMONT', 'LUDLOW', 'MAMARONECK', 'MANITOU', 'MARBLE HILL', 'MEADOWLANDS SPORTS COMPLEX', 'MELROSE', 'MERRITT 7', 'MILFORD', 'MORRIS HEIGHTS', 'MOUNT KISCO', 'MOUNT PLEASANT', 'MT VERNON EAST ', 'MT VERNON WEST', 'NAUGATUCK', 'NEW CANAAN', 'NEW HAMBURG', 'NEW HAVEN', 'NEW ROCHELLE', 'NH-STATE ST.', 'NOROTON HEIGHTS', 'NORTH WHITE PLAINS', 'OLD GREENWICH', 'OSSINING', 'PATTERSON', 'PAWLING', 'PEEKSKILL', 'PELHAM', 'PHILIPSE MANOR', 'PLEASANTVILLE', 'PORT CHESTER', 'POUGHKEEPSIE', "PURDY'S", 'REDDING', 'RIVERDALE', 'RIVERSIDE', 'ROWAYTON', 'RYE', 'SCARBOROUGH', 'SCARSDALE', 'SEYMOUR', 'SOUTH NORWALK', 'SOUTHEAST', 'SOUTHPORT', 'SPRINGDALE', 'SPUYTEN DUYVIL', 'STAMFORD', 'STRATFORD', 'TALMADGE HILL', 'TARRYTOWN', 'TENMILE RIVER', 'TREMONT', 'TUCKAHOE', 'UNIVERSITY HEIGHTS', 'VALHALLA', 'WAKEFIELD', 'WASSAIC', 'WATERBURY', 'WEST HAVEN', 'WESTPORT', 'WHITE PLAINS', 'WILLIAMS BRIDGE', 'WILTON', 'WOODLAWN', 'YANKEES-E153 ST.', 'YONKERS']
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.support.ui import Select
  

• Browser Snapshot: