Webscraping with Selenium using Xpath

Question

I'm looking for some numbers on https://tnnslive.com/

The numbers I'm looking for on tnnslive.com:

the numbers I'm looking for on tnnslive.com

If I copy the xpath in Chrome, I get:

//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]

If I run the following code:

from selenium import webdriver

# Start a new Chrome browser session
driver = webdriver.Chrome(executable_path="PATH_TO_CHROMEDRIVER")

# Navigate to the desired URL
driver.get('https://tnnslive.com/match/VDDGj17PodORnq62YwQk')

# Find the element by its XPath and print its text
element = driver.find_element("xpath", '//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]')    

# Close the browser
driver.quit()

the error occurs:

NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]"}
  (Session info: chrome=115.0.5790.171)

Why? How can I solve this issue?

score 0 · Accepted Answer · answered Aug 09 '23 at 20:35

To print the text associated with 1st Serve Points Won for the first player within the website you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

Using XPATH, following-sibling and text attribute:

driver.get(url='https://tnnslive.com/match/MpVVddDZigie7GHM9ip8')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[text()='1st Serve Points Won']//following-sibling::div[1]"))).text)

Using XPATH, following and get_attribute("innerHTML"):

driver.get(url='https://tnnslive.com/match/MpVVddDZigie7GHM9ip8')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[text()='1st Serve Points Won']//following::div[1]"))).get_attribute("innerHTML"))

Console output:
```
64% (29/45)
```

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

References

Link to useful documentation:

get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

It is better to use just one slash instead of two slashes `//` before `following` — Siebe Jongebloed, Aug 10 '23 at 06:06

Webscraping with Selenium using Xpath

1 Answers1

References