How can I get Selenium WebDriver to wait until
element contains data before proceeding?

Question

Am trying to automate feeding a url to Pingdom's Website Speed Test (see http://tools.pingdom.com/fpt/) and then extract and print the result of that test.

I have written some code but I can't figure out how to get the data out of the 'Perf. grade' element.

It seems that the element exists before the test is run (which I guess is run server-side?) but is empty. Then once the test has finished the element is populated with the value.

How can I get Selenium to wait until this value is populated before attempting to print it?

Here is my code:

import datetime
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Pingdom Website Speed Test
for i in range(0, 1):

    # Initialises chromedriver
    driver = webdriver.Chrome(executable_path=r'C:\Users\Desktop\Python\chromedriver\chromedriver.exe')

    # Opens Pingdom homepage
    driver.get('http://tools.pingdom.com/fpt/')

    # Looks for search box, enters 'http://www.url.com/' and submits it
    pingdom_url_element = driver.find_element_by_id('urlinput')
    pingdom_url_element.send_keys('http://www.url.com/')
    pingdom_test_button_element = "//button[@tabindex='2']"
    driver.find_element_by_xpath(pingdom_test_button_element).click()

    # Waits until page has loaded then looks for attribute containing the report score's value and returns the value
    pingdom_performance_result = WebDriverWait(driver, 60).until(EC.presence_of_element_located((By.XPATH, "//div[@id='rt_sumright']/dl[@class='last']/dd[1]")))

    print('Pingdom score:')
    print(datetime.datetime.now().strftime("%d-%m-%Y %H:%M:%S"), "---", pingdom_performance_result.text)

    driver.close()

    i += 1

score 4 · Accepted Answer · edited May 23 '17 at 11:59

You can make a custom expected condition and wait for the grade to have a value - or, to match a specific regular expression in this case:

from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC

class wait_for_text_to_match(object):
    def __init__(self, locator, pattern):
        self.locator = locator
        self.pattern = pattern

    def __call__(self, driver):
        try:
            element_text = EC._find_element(driver, self.locator).text
            return self.pattern.search(element_text)
        except StaleElementReferenceException:
            return False

Usage:

import re

wait = WebDriverWait(driver, 60)
pattern = re.compile(r"\d+/\d+")
pingdom_performance_result = wait.until(wait_for_text_to_match((By.XPATH, "//div[@id='rt_sumright']/dl[@class='last']/dd[1]"), pattern))

print(pingdom_performance_result.text)

Great, thanks for this solution, it works pretty well! The only minor thing is that PyCharm tells me that `StaleElementReferenceException` is an unresolved reference. Any idea why? — chewflow, May 18 '16 at 08:02

How can I get Selenium WebDriver to wait until element contains data before proceeding?

1 Answers1

How can I get Selenium WebDriver to wait until
element contains data before proceeding?