Cannot find table element from div element in selenium python

Question

I am trying to select a table values from div element using selenium,

When I inspect the element, I could see the element contains the table but I cannot find it while viewing the source code (which is happening while trying to access through selenium).

During Inspect:

Below is the code I tried,

totals = driver.find_element_by_id("totals")#gets the totals div element. Only one is there
labels=totals.find_elements_by_class_name("hasLabel")#should get the table rows, but it was empty
print(len(labels))#returned 0

But, It was empty (as I saw in the source code). I need to export those values in a file as part of automation

Is there any other way I can extract those due values?.

Answer 1

Your observation seems to be near perfect. As per the HTML within During Inspect snapshot which you have shared, The element with text as Taxes Due is highlighted.

---

While within the While viewing the source code snapshot which you have extracted through Selenium, possibly the Page Source was pulled out even before the child elements have completely rendered within the DOM Tree:

Hence the child elements within <div id='totals'></div> are not present withi the Page Source.

---

The relevant HTML in text format would have help to construct a canonical answer. However, to get the total number of <tr> elements you need to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• Using XPATH:

  print(len(WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//table/tbody//tr[@class='hasLabel'][./td[@class='label']]")))))
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC