Selected element is Null using selenium

Question

I want to achieve a dynamic waiting time when crawling data.

The HTML looks like this:

<tr class="mv-quote-row" style="border-top-width: 1px;">
 <td style="text-align: left;">Week 34/21</td>
 <td style="text-align: right;">-</td>
 <td style="text-align: right;">-</td>
 <td style="text-align: right;">15</td>
 <td class="mv-quote-button mv-button-image mv-hyperlink-button">
  <svg class=" mv-quote-inline-chart" viewBox="0 0 16 16">
   <path d="M9,8,5,4,0,10.18V16H16V0Zm6,7H1V10.54l4.08-5,4,4L15,2.66Z"></path></svg>
 </td>
</tr>

The thing is I want to crawl the rows inside a table, Html here shows one row inside it. The row originally looks like 'week 34/21' '-' '-' '15', 4 columns. When crawling, It usually returns Null value for the last three bits. But when I set a static waiting time long enough,like time.sleep(). It is able to crawl the whole row of data.

row_a = WebDriverWait(driver, 20).until(
     EC.presence_of_element_located((By.XPATH, "//*[@id=\"baseloadwidget_jfk\"]/table/tbody/tr[1]/td[3]"))
 )
row_b = WebDriverWait(driver, 20).until(
    EC.visibility_of_element_located((By.XPATH, "//*[@id=\"baseloadwidget_jfk\"]/table/tbody/tr[1]/td[3]"))
)

I tried the presence_of_element_located and visibility_of_element_located method above, but not working. So I checked the website, which showed the loading order goes as, the data table - the 'week' column - the other three bits data. I think the reason could be the element is existed but the value is still loading, thus it returns null value.

In this situation, how can I make the waiting time dynamic? Like, add a condition returning the element until the text is not null? Thanks so much

Apply Explicit wait in such a way that it waits until the element is not `Null` or empty. Like this [Link](https://stackoverflow.com/a/53405405/16452840) — pmadhu, Aug 27 '21 at 06:58

score 1 · Accepted Answer · answered Aug 27 '21 at 08:45

If Explicit waits is not working, which ideal should work for dynamic content loading. It could be due to rows locators are not correct. or may be you first try to store the elements in a list.

if all the rows are in Selenium view port, you could try this code :-

tr_td_list = driver.find_elements(By.XPATH, "//tr[@class='mv-quote-row']//td")
print(len(tr_td_list)) # this should print 16

for ele in tr_td_list:
  print(ele.text)

also you can implement actions chain to move to particular tr and tds:

tr_td_list = driver.find_elements(By.XPATH, "//tr[@class='mv-quote-row']//td")
print(len(tr_td_list)) # this should print 16

for ele in tr_td_list:
  ActionChains(driver).move_to_element(ele).perform()
  print(ele.text)

Imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains

Selected element is Null using selenium

1 Answers1