Best way for Selenium to wait for elements that are not always present

Question

I'm writing a simple python/selenium code to extract data from a url. I would like to know what is the best way to wait for a group of elements to be visible, like this:

Element A: A dialog that randomly appears on the website. If the element is there, it must be closed/clicked. If not, ignored.
Element B: Article price. If the article is not free, the element is  present on the website.
Element C: Article price. If the is free, the element is present on the website
Element D: Shipping time (sometimes it is inside an iframe)
Element E: Shipping time (other time is is outside the iframe)

Sometimes, the elements take a while to load, so I've added a time.sleep(1.5). I know that selenium have a expected_conditions wait, but that works for a element that is always present.

This is the code I have so far, but I don't think it is good enought as it used an implicit wait. How can I improve this code?

try:
    time.sleep(1.5)
    try:
        #ElementA
        driver.find_element(By.CSS_SELECTOR, "div.andes-dialog__wrapper div.andes-dialog__container a.andes-dialog_-close").click()  
    except: pass
    try:
        #Element B: Article price in iframe (not free)
        shipping_price=driver.find_element(By.CSS_SELECTOR, "div.ui-shipping_column.ui-shipping_price span.price_amount").text
    except:
        #Element C: Article price in iframe (free)
        shipping_price=driver.find_element(By.CSS_SELECTOR, "div.ui-shipping__column.ui-shipping_price").text
    #Element D: Shipping time in iframe
    shipping_time=driver.find_element(By.CSS_SELECTOR, "div.ui-shipping_column.ui-shipping_description span.ui-shipping__title").text
except Exception as e:
    #Element E: Shipping time outside iframe
    shipping_time=driver.find_element(By.CSS_SELECTOR, "form.ui-pdp-buybox div.ui-pdp-shipping p.ui-pdp-media__title").text

This helps: https://stackoverflow.com/questions/35813006/selenium-wait-until-one-of-the-two-elements-is-present — Cristian Avendaño, Oct 14 '21 at 18:29

Shine J · Answer 1 · 2021-10-14T16:43:09.240

0

You should you WebDriverWait like this:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome("D:/chromedriver/94/chromedriver.exe")

driver.get("https://www.website.com")
# wait 60 seconds 
wait = WebDriverWait(driver,60)
try:
    wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.ui-shipping_column.ui-shipping_price span.price_amount")))
    shipping_price=driver.find_element(By.CSS_SELECTOR, "div.ui-shipping_column.ui-shipping_price span.price_amount").text
except Exception as ex:
    # do something
    print(ex)

edited Oct 14 '21 at 16:43

answered Oct 14 '21 at 16:37

Shine J

798
1
6
11

But If I do that.. the driver will wait 60 seconds for each element.. and if the element is not always there that can be very slow. – Cristian Avendaño Oct 14 '21 at 17:08
I found this: https://codoid.com/selenium-expectedconditions-with-logical-operators/ – Cristian Avendaño Oct 14 '21 at 17:12

Best way for Selenium to wait for elements that are not always present

1 Answers1