0

i'm fairly new to selenium and i'm building a scraper to extract info from a table.

I'm able to acces the table body by ID with no problem but when I try to access it's children they are not found.

the inspector shows the xpath for the first cell as //*[@id="tb_list"]/tr[1]/td[1] but

find_element_by_xpath(//*[@id="tb_list"]/tr[1]/td[1])

can't find it.

I also tried the following to no avail.

table = driver.find_element_by_id("tb_list")
table.find_element_by_xpath(".//tr[1]/td[1]")

it's able to find tb_list but fails to locate the children

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":".//tr[1]/td[1]"}

Everywhere I looked people suggest one of these 2 methods, what am I doing wrong? The table is dynamically populated from a database, could this be an issue?

I'm using python and the chrome web driver, I'm hesitant to give a snippet of the html as the site is not publicly available and i dont own it.

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
javi8
  • 33
  • 3

1 Answers1

0

[1] indicates first descendent. So the xpath:

//*[@id="tb_list"]/tr[1]/td[1]

can be optimized as:

//*[@id="tb_list"]/tr/td

Effectively the line of code would be:

driver.find_element(By.XPATH, "//*[@id='tb_list']/tr/td")

Ideally, to locate the element you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

  • Using XPATH:

    element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='tb_list']/tr/td")))

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352