How to select a dropdown menu using Selenium Python

Question

I'm working on a personal project, trying to use selenium to web scrape my locals teams results from a website. The site has three drop-down lists, at the moment I'd be happy to just be able to alter one. The code below clicks reject cookies when I enter the page, that's what the WebDriver line does. I'm getting as far as 'dropdown' and then an error that no such element exists. I've tried CSS_SELECTOR, XPATH and all the other options to no avail. I am only new to this so it could be something simple I'm missing but I've read a lot of the previous forums with similar questions and the answers didn't work for me.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
import time

driver = webdriver.Firefox()
driver.get("https://www.leinsterrugby.ie/domestic-rugby/domestic-fixtures-results/")

time.sleep(3)
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='onetrust-reject-all-handler']"))).click()
time.sleep(3)

dropdown=Select(driver.find_element(By.XPATH,value='//*[@id="groups_data"]')
time.sleep(5)
dropdown.select_by_value('5756')

driver.quit() # close browser

_The code below clicks reject cookies_: At my end the cookie popup remains folded and doesn't expands. Isn't it the expected behaviour at your end? — undetected Selenium, Jul 07 '22 at 16:40
_I'm getting as far as 'dropdown'_: With which dropdown are you trying to interact? — undetected Selenium, Jul 07 '22 at 16:41
Does this answer your question? [Switch to an iframe through Selenium and python](https://stackoverflow.com/questions/44834358/switch-to-an-iframe-through-selenium-and-python) — DonnyFlaw, Jul 07 '22 at 16:56

score 1 · Accepted Answer · answered Jul 07 '22 at 16:55

The element with dropdown options as Men, Schools, Women, Youth is within an <iframe> so you have to:

Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.

You can use either of the following Locator Strategies:

Using CSS_SELECTOR:

driver.get('https://www.leinsterrugby.ie/domestic-rugby/domestic-fixtures-results/')
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src='https://shared2.sportsmanager.ie/~leinsterrugby/']")))
Select(WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "select[name=user]")))).select_by_value('7183')

Using XPATH:

driver.get('https://www.t-online.de/themen/e-mail')
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[@src='https://shared2.sportsmanager.ie/~leinsterrugby/']")))
Select(WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//select[@name='user']")))).select_by_value('7183')

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Browser Snapshot:

Thanks! Yeah my issue was that I wasn't aware it was in an iframe, thanks so much your answer is really helpful. — Aaron Ryan, Jul 08 '22 at 09:55

Michael Mintz · Answer 2 · 2022-07-09T16:05:40.597

Many of those lines can be simplified by using SeleniumBase.

Here's code that searches for results on that site and then prints the results: (self.switch_to_frame() is needed to switch to the iframe, and self.select_option_by_text() is for the dropdowns.)

from seleniumbase import BaseCase

class RugbyResultsTests(BaseCase):
    def test_get_rubgy_results(self):
        self.open("https://www.leinsterrugby.ie/domestic-rugby/domestic-fixtures-results/")
        self.switch_to_frame('iframe[src*="sportsmanager.ie/~leinsterrugby/"]')
        self.sleep(1)
        self.select_option_by_text("select#usercompyear", "2020-2021")
        self.sleep(3)
        self.select_option_by_text("select#groups_data", "Leinster Seconds League (J2)")
        self.sleep(1)
        self.click('a[href*="sportsmanager.ie/~leinsterrugby/league/155189"]')
        self.sleep(2)
        print("\n" + self.get_text("table.table-condensed"))

Output: (After running the above test with pytest with seleniumbase installed via pip.)

POS TEAM PLD W D L PF PA DIFF BP BP L DED PTS
1 Ashbourne 1 1 0 0 18 17 1 0 0 0 4
2 Gorey 1 1 0 0 0 0 0 0 0 0 4
3 Kilkenny 1 0 0 1 17 18 -1 1 1 0 1
4 Suttonians 0 0 0 0 0 0 0 0 0 0 0
5 Monkstown 0 0 0 0 0 0 0 0 0 0 0
6 Seapoint 0 0 0 0 0 0 0 0 0 0 0
7 Bective Rangers 0 0 0 0 0 0 0 0 0 0 0
8 Dundalk 1 0 0 1 0 0 0 0 0 0 0

The default browser for SeleniumBase tests is Chrome, but you can change that by calling pytest --firefox.

Thanks, your output looks great. I've never even heard of seleniumbase and haven't done a lot of pytest so I'm not sure where to start on your solution. But I appreciate how good your answer looks. — Aaron Ryan, Jul 11 '22 at 10:47

How to select a dropdown menu using Selenium Python

2 Answers2