Selenium, Presence of one of many elements located?

Question

Building off of the answer to How to wait until the page is loaded with Selenium for Python? I am attempting to create a method that allows multiple elements to be polled for presence using Expected Conditions.

I receive an error 'bool' object is not callable on the line containing: wait.until(any(expectations)).

The thought process was to allow an number of Xpaths to be passed as expected conditions, then using any(), in a similar manner to this java based answer, Trying to wait for one of two elements in the page using selenium xpath, check if any of the conditions are present.

What would be the proper way use any() in this case? Or better yet, what needs to be done for this method to work?

Assume that the Selenium .get('url') has already been executed immediately prior to calling this method.

def wait_with_xpath_expectation(self, search_elements, timeout=6, max_attempts=3):
    """
    Selenium wait for an element(s) designated by Xpath to become available in the DOM. Useful for javascript AJAXy
    loading pages where content may be be rendered dynamically on the client side after page load appears complete.
    search_elements may be one Xpath as a string or many as a list. This allows for any of multiple elements on a
    page or pages to be determined to have loaded based on expectations.
    :param search_elements: string or list (strings converted to lists), Xpath(s)
    :param timeout: int, seconds
    :param max_attempts: int, time to wait before giving up on page loading
    :return: Boolean, True if page has loaded, False if all attempts have failed
    """

    # Page is not loaded yet
    loaded = False

    # Initialize attempt count
    attempt = 0

    # If only one element has been passed, ensure it is encapsulated by a list
    if type(search_elements) is str:
        search_elements = [search_elements]

    # Begin the polling loop
    while attempt < max_attempts:

        try:

            while loaded is False:
                # Create a list of expected elements using list comprehension
                expectations = [EC.presence_of_element_located((By.XPATH, element)) for element in search_elements]

                # Define wait
                wait = WebDriverWait(self.browser, timeout)

                # Execute
                wait.until(any(expectations))

                # Acknowledge load
                loaded = True

                print("Success: Page loaded based on expectations")

                # Exit returning True for load
                return loaded

        except TimeoutException as e:

            # Increment attempts
            attempt += 1

            # Check again if max attempts has not been exceeded
            while attempt < max_attempts:

                # Increase timeout by 20%
                timeout *= .2

                # Try again 
                continue

            # Print an error if max attempts is exceeded
            print("Error: Max load with expectations attempts exceeded,", e)

            # Exit returning False for load
            return loaded

Additional research turned up the fact that I could perform a union of nodes and combine the Xpath addresses with | into one Xpath. https://stackoverflow.com/questions/5350666/xpath-or-operator-for-different-nodes. Still working on a solution. — Liquidgenius, Aug 01 '18 at 20:46

score 8 · Accepted Answer · answered Aug 04 '18 at 06:09

8

You can have an expected condition class to wait for a combination of expected conditions. Here is an example of one.

class wait_for_all(object):
    def __init__(self, methods):
        self.methods = methods

    def __call__(self, driver):
        try:
            for method in self.methods:
                if not method(driver):
                    return False
            return True
        except StaleElementReferenceException:
            return False

This would then be used by building an array of the expected conditions and checking for all of them in the same wait. (Example lines split for clarity.)

methods = []
methods.append(EC.visibility_of_element_located(BY.ID, "idElem1"))
methods.append(EC.visibility_of_element_located(BY.ID, "idElem2"))
method = wait_for_all(methods)
WebDriverWait(driver, 5).until(method)

This will perform one five second wait while checking for visibility of two different elements.

I have documented this further in a blog post here.

answered Aug 04 '18 at 06:09

Tim Butterfield

286
1
3

Hi Tim. I like this implementation much better than mine, thank you! Multiple DOM elements can be waited for explicitly instead of a union. On top of that it looks very easy to use. The one item I'd change is BY.ID to BY.XPATH as I've found that some sites do not properly use class and id tags, but XPATH is always present due to the DOM. Accepting answer, thank you. – Liquidgenius Aug 15 '18 at 16:39
Thanks. I just used By.ID as example element locator type. Since each expected condition is created separately, you can combine different locator types as appropriate for that element. One element may be located By.ID, another could use By.XPATH, By.NAME, By.CLASS_NAME, By.TAG_NAME, By.CSS_SELECTOR, etc. – Tim Butterfield Aug 16 '18 at 18:24
Accepting answer – Liquidgenius Nov 01 '18 at 20:19
This is perfect answer but now selenium expected condition function visibility_of_element_located takes argument as 1 tuple. – Selim Mıdıkoğlu Aug 15 '20 at 06:47
I have a similar problem to the question, and this answer looked very promising, but when I try it, it only seems to work if ALL the elements in the list are located (boolean AND). But my understanding of the original question is that there should be a positive match if ANY of the elements are located (boolean OR). I might have a mistake in my code, but I just wanted to confirm that we're trying to solve the same problem (you wrote "wait for a combination of expected conditions", which leaves it open to interpretation). Btw, the link to your blog doesn't seem to work anymore. – Ratler Feb 08 '23 at 00:53
@Ratler The functionality of Selenium has changed since I had originally created this solution. If it can still be made to work, it would need to be updated for a more current version of Selenium. (Yep. The blog is offline.) – Tim Butterfield Feb 09 '23 at 15:49
@TimButterfield, upon further inspection, I realised that this happens because the way you have it at the moment, the `for` loop in `__call__` returns False on first mismatch; what we're looking for is a True on first match, False otherwise. Beyond that, I think your solution is ideal, so instead of submitting a new answer, I've edited yours accordingly, but it needs to be approved. Perhaps you can have a look and see if you agree with my changes. – Ratler Feb 11 '23 at 11:39

score 1 · Answer 2 · answered Aug 01 '18 at 20:52

Once I realized Xpath has the ability for unions, the following method works. I will leave this answer open in case anyone else has a better one.

def wait_with_xpath_expectation(self, search_elements, timeout=6, max_attempts=3):
    """
    Selenium wait for an element designated by Xpath to become available in the DOM. Useful for javascript AJAXy
    loading pages where content may be be rendered dynamically on the client side after page load appears complete.
    search_elements may be one Xpath as a string or many as a list. This allows for any of multiple elements on a
    page or pages to be determined to have loaded based on expectations.
    :param search_elements: string or list (strings converted to lists), Xpath(s)
    :param timeout: int, seconds
    :param max_attempts: int, time to wait before giving up on page loading
    :return: Boolean, True if page has loaded, False if all attempts have failed
    """

    # Page is not loaded yet
    loaded = False

    # Initialize attempt count
    attempt = 0

    # If only one element has been passed, ensure it is encapsulated by a list
    if type(search_elements) is str:
        search_elements = [search_elements]

    # Begin the polling loop
    while attempt < max_attempts:

        try:

            while loaded is False:
                # Decompose the list of Xpaths to a union of nodes
                node_union = " | ".join(search_elements)

                expectation = EC.presence_of_element_located((By.XPATH, node_union))

                # Define wait
                wait = WebDriverWait(self.browser, timeout)

                # Execute
                wait.until(expectation)

                # Acknowledge load
                loaded = True

                print("Success: Page loaded based on expectations")

                # Exit returning True for load
                return loaded

        except TimeoutException as e:

            # Increment attempts
            attempt += 1

            # Check again if max attempts has not been exceeded
            while attempt < max_attempts:

                # Increase timeout by 20%
                timeout *= .2

                # Try again
                continue

            # Print an error if max attempts is exceeded
            print("Error: Max load with expectations attempts exceeded,", e)

            # Exit returning False for load
            return loaded

Selenium, Presence of one of many elements located?

2 Answers2

Linked