Obtaining option tag attributes under select tag without an ID

Question

I need to obtain the attributes of an option under a select tag. However, i am unable to locate the select tag since the select tag does not have an id.

<div class="dropdown-wrapper">
   <div class="mobile-dropdown">
   <span class="mobile-arrow"></span>
     <select>
        <option data-url="/series/17948/commentary/1115802/new-zealand-vs- 
        pakistan-1st-odi-pak-in-nz-2017-18?innings=1" value="NZ Innings">NZ 
       Innings
        </option>
        <option data-url="/series/17948/commentary/1115802/new-zealand-vs- 
        pakistan-1st-odi-pak-in-nz-2017-18?innings=2" value="PAK 
        Innings">PAK Innings</option>
    </select>
  </div>
</div>

i tried two ways

1. Locating the select tag using find_elements_by_tag_name('select'), then obtaining all text within the select element, then locating the option tag using find_elements_by_xpath ("//option[contains(text(), text)]"). Once i located the option tag, i could use get_attributes to the required attribute. Not only does this seem very complex, it also doesn't work sometimes as it doesn't give the option text.

2. I tried using Select by using Select(find_element_by_css_selector("class"). The class name used is from the div tag. Then used select.select_by_index(1).getattribute(). However, i got an error "Select not defined".

1st Code

elem=driver.find_elements_by_tag_name('select')
options=[x.text for x in elem]
first_inn=options[2].split('\n')[1]
second_inn=options[2].split('\n')[0]
option=driver.find_elements_by_xpath("//option[contains(text(), first_inn)]")
option[7].get_attribute('data-url')

2nd Code

  select = Select(driver.find_element_by_css_selector("mobile-dropdown"))
    first_inn=select.select_by_index(1).get_attribute('data-url')
    second_inn=select.select_by_index(0).get_attribute('data-url')

For the 1st code, i am getting ['','',''] and for the second code i am getting an error message "name 'Select' is not defined"

Answer 1

Try with this xpaths:

for NZ Innings: (.//div[@class='mobile-dropdown']/select/options)[1]

for PAK Innings: (.//div[@class='mobile-dropdown']/select/options)[2]

Answer 2

Update with usage example:

teams = []
teams_options = driver.find_elements_by_xpath("(//div[@class='mobile-dropdown'])[1]//select//option")
for option in teams_options:
    teams.append({"name": option.get_attribute("value"), "url": option.get_attribute("data-url")})

# print teams
print(teams[0].get("name"), teams[0].get("url"))
print(teams[1].get("name"), teams[1].get("url"))
# or
for team in teams:
    print(team.get("name"), team.get("url"))

commentaries = {}
commentary_options = driver.find_elements_by_xpath("(//div[@class='mobile-dropdown'])[2]//select//option")
for option in commentary_options:
    commentaries[option.get_attribute("value")] = option.get_attribute("data-url")

# print and commentaries dict usage example
print(commentaries.get("Full commentary"))
print(commentaries.get("Wickets"))
print(commentaries.get("Boundary"))
print(commentaries.get("Highlights"))

# print all commentaries
for commentary in commentaries.keys():
    print(commentary, commentaries.get(commentary))

Using xpath:

first_select_options = driver.find_elements_by_xpath("(//div[@class='mobile-dropdown'])[1]//select//option")
for option in first_select_options:
    print(option.get_attribute("value"), option.get_attribute("data-url"))

second_select_options = driver.find_elements_by_xpath("(//div[@class='mobile-dropdown'])[2]//select//option")
for option in second_select_options:
    print(option.get_attribute("value"), option.get_attribute("data-url"))

Using css selector:

selects = driver.find_elements_by_css_selector(".match-commentary select")
for s in selects:
    for option in s.find_elements_by_tag_name("option"):
        print(option.get_attribute("value"), option.get_attribute("data-url"))

Using WebDriverWait:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

#..    

all_selects = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".mobile-dropdown select")))
for option in all_selects[0].find_element_by_tag_name("option"):
    print(option.get_attribute("value"), option.get_attribute("data-url"))

for option in all_selects[1].find_element_by_tag_name("option"):
    print(option.get_attribute("value"), option.get_attribute("data-url"))

Answer 3

To extract the <option> tag attributes from the second option e.g. <option> with text as PAK Innings within the <select> tag you have to induce WebDriverWait for the element_to_be_clickable() and you can use the following Locator Strategies:

• CSS_SELECTOR:

  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.dropdown-wrapper>div.mobile-dropdown select"))).click()
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.dropdown-wrapper>div.mobile-dropdown select option:nth-child(2)"))).get_attribute("data-url"))
  

• XPATH:

  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[@class='dropdown-wrapper']/div[@class='mobile-dropdown']//select"))).click()
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='dropdown-wrapper']/div[@class='mobile-dropdown']//select//following-sibling::option[2]"))).get_attribute("data-url"))