How to use the WebDriverWait() to access the value of the next element having the same class names in selenium python

Question

I am trying to access the values of both the fields Year and Quarter from this particular site. With the help from one member of StackOverflow, I was able to implement the code for the year part, now if I want to access the quarter part so how can I access the same.

Below is the implementation so far.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-error')
options.add_argument('--ignore-ssl-errors')

url = "https://lifeinsurance.adityabirlacapital.com/about-us/public-disclosure"
driver = webdriver.Chrome(executable_path='drivers/chromedriver.exe')

driver.get(url)

WebDriverWait(driver, 20).until(
    EC.element_to_be_clickable((By.CSS_SELECTOR, "div.selectize-input.items.full.has-options.has-items"))
).click()

year_dropdown = WebDriverWait(driver, 20).until(
    EC.visibility_of_all_elements_located(
        (By.CSS_SELECTOR, "div.selectize-dropdown-content div.option")
    )
)

for year in year_dropdown:
    print(year.text)

Any hints and suggestions are welcome.

Sers · Accepted Answer · 2021-02-12T07:24:47.413

You have same structure for both years and quarters. One with selectPublicYear and another with selectPublicQuarter class

wait = WebDriverWait(driver, 20)

# Years
year_selector = ".selectize-control.selectPublicYear"
year_option_selector = year_selector + " .option"

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, year_selector))).click()

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, year_option_selector)))
years = driver.find_elements(By.CSS_SELECTOR, year_option_selector)
for year in years:
    print(year.text)
years[-1].click()

# Quarters
quarter_selector = ".selectize-control.selectPublicQuarter"
quarter_option_selector = quarter_selector + " .option"

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, quarter_selector))).click()
# Wait for option element to be clickable.
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, quarter_option_selector)))
# Get all option elements
quarters = driver.find_elements(By.CSS_SELECTOR, quarter_option_selector)
for quarter in quarters:
    print(quarter.text)
# Select last one
quarters[-1].click()

You can add method to select dropdown by value:

def select_dropdown(name, selector, value):
    print(f'Select "{name}" dropdown, value: "{value}"')

    # Selectors
    selector = ".selectize-control" + selector
    option_selector = selector + " .option"

    wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, selector))).click()

    # Wait for option element to be clickable
    wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, option_selector)))

    # Get all options elements
    option_elements = driver.find_elements(By.CSS_SELECTOR, option_selector)
    # Get text values from options
    options = [opt.text.strip() for opt in option_elements]
    print(f'Available values: {options}')

    # Check if options have value
    if value not in options:
        raise Exception(f'"{name}" dropdown does not have "{value}" value. Available values: {options}')

    option_elements[options.index(value)].click()

Call method:

select_dropdown(name="Year", selector=".selectPublicYear", value="2020-2021")
select_dropdown(name="Quarter", selector=".selectPublicQuarter", value="Q2: Sep, 2020")

I followed your implementation and it works for printing the value but when try to use `quarter.click()` it throws `StaleElementReferenceException` but it does not gives this error for the year section. Can you tell me why is that so and how can I fix it. — Shaikh Abuzar, Feb 11 '21 at 21:46
You can check update code. Selected successfully year and quarter. — Sers, Feb 12 '21 at 07:29
When I tried to select `Q1: Jun, 2020` it gave the same error. — Shaikh Abuzar, Feb 12 '21 at 09:25
I fixed the issue by adding a delay of 10secs. By the way thanks for all the help — Shaikh Abuzar, Feb 12 '21 at 09:36

score 0 · Answer 2 · answered Feb 12 '21 at 07:38

To extract the texts e.g. Q4: Mar, 2021, from all of the <class="option"> using Selenium and python you have to induce WebDriverWait for visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

Using XPATH:

driver.get('https://lifeinsurance.adityabirlacapital.com/about-us/public-disclosure')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//label[text()='Quarter']//following::div[@class='selectize-input items full has-options has-items']"))).click()
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@class='selectize-dropdown-content']//div[contains(@class, 'option')]")))])

Console Output:

['All Quarters', 'Q4: Mar, 2021', 'Q3: Dec, 2020', 'Q2: Sep, 2020', 'Q1: Jun, 2020']

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

How to use the WebDriverWait() to access the value of the next element having the same class names in selenium python

2 Answers2