0

I am using selenium to retrieve data from a web page. I use expected_conditions selenium module in order to make the code more robust and to avoid having long sleeps in order to make sure that the data is loaded and can be used. Unfortunately I still got sporadic failures and I must use sleep() calls to get the data I want.

Is there an elegant, 100% sure solution to know if a web element is not only present and visible, but it is also "final"?

The following code shows a bit better what my problem is:

Instead of visibility, I have used also presence of the element but it was the same or a bit worse.

def check_id(driver):
    try:
        id_element = WebDriverWait(driver, 10).until(
            expected_conditions.visibility_of_element_located((By.ID, "whatever-id")))
        print(f"ID checkpoint before nap: {id_element.text} ")
        time.sleep(2)
        id_element = driver.find_element(By.ID, "whatever-id")
        print(f"ID checkpoint after nap: {id_element.text} ")
    except Exception:
        print("Exception")



Output: (on a sunny day)
ID checkpoint before nap: MY_EXPECTED_ID 
ID checkpoint after nap: MY_EXPECTED_ID 

        
Output: (on a rainy day)
ID checkpoint before nap:  
ID checkpoint after nap: MY_EXPECTED_ID
Tompa Hawk
  • 3
  • 3

2 Answers2

0

EC.presence_of_element_located:

An expectation for checking that an element is present on the DOM of a page. This does not necessarily mean that the element is visible.

EC.visibility_of_element_located:

An expectation for checking that an element is present on the DOM of a page and visible. Visibility means that the element is not only displayed but also has a height and width that is greater than 0.

EC.element_to_be_clickable:

An Expectation for checking an element is visible and enabled such that you can click it.

Suggestions:

  • If visibility_of_element_located method isn't working properly, worth trying with element_to_be_clickable
  • Also I see the wait time you have specified is 10s, try increasing it to 30s
Shawn
  • 4,064
  • 2
  • 11
  • 23
  • Thanks! Is element_to_be_clickable usable when it is a text element? It is not expected that someone clicks on it, it is just a field with text. I have started with a 30 sec timeout, but it is not really relevant, the result is the same as the execution continues as the visibility is given and it is not neccessarily means that the data is loaded. – Tompa Hawk Jun 16 '23 at 10:06
  • Yes `element_to_be_clickable` can be used on any web element not necessarily have to be clickable. It basically checks if the element is visible and not disabled. I know the naming of the method is little misleading. If you ask me it makes more sense to name this method as `element_to_be_visible_and_not_disabled` – Shawn Jun 16 '23 at 10:18
  • It gives me the same result, 1-2 out of 10 executions result empty string after the condition. (timeout is 30, element is clickable) I think I cannot avoid writing a wait until clicakble and not empty function. – Tompa Hawk Jun 16 '23 at 11:02
0

The following function seemed to solve the problem, but I am sure that someone can come up with something nicer. Please let me know what should I do differently. I am new to web deveolpment so I might do something very wrong with this approach.

def find_element_with_data(driver, strategy, searchstring, can_be_empty=True, timeout=5):
    try:
        element = WebDriverWait(driver, 30).until(
            expected_conditions.element_to_be_clickable((strategy, searchstring)))
        print(f"Checkpoint before nap: {element.text} ")
        if can_be_empty:
            return element
        for loop_cnt in range(timeout):
            time.sleep(1)
            element = driver.find_element(strategy, searchstring)
            print(f"Checkpoint after {loop_cnt} seconds nap: {element.text} ")
            if element.text:
                return element
        return None
    except Exception as exc:
        print(f"Exception while finding web element: {exc}")
        return None

# Calling
my_id_element = find_element_with_data(By.ID, "whatever-id", can_be_empty=False, timeout=2)
Tompa Hawk
  • 3
  • 3