How can I extract the text from a webelement using selenium

Question

Here I have tried to access leetcode question title, which are available on the given link. However I seem to not be able to access the text in the specified web element. The error I get says:

AttributeError: 'list' object has no attribute 'text'

I also get a warning:

service = Service(executable_path=gecko, log_path='geckodriver.log')
DeprecationWarning: log_path has been deprecated, please use log_output

Please help me fix these, since I cannot find any solution which does not return the same error.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.service import Service

# URL of the page with questions
url = 'https://leetcode.com/problemset/all/?page=1&topicSlugs=array'
gecko= 'path'
i = 3
# XPath expression for the elements containing the text
xpath_expression = f"/html/body/div[1]/div/div[2]/div[1]/div[1]/div[5]/div[2]/div/div/div[2]/div[1]/div[2]/div/div/div/div/a"
firefox_path="path"

options = Options()
service = Service(executable_path=gecko, log_path='geckodriver.log')
driver = webdriver.Firefox()
driver.get(url)
element = driver.find_elements(By.XPATH, xpath_expression)
element_text = element.get_attribute('text')
print(element_text)
driver.quit()

Answer 1

The web element returned is a list, so this should work in your case

element_text = element[0].text

but ideally, a list implies several elements so would make sense to iterate over

elements = driver.find_elements(By.XPATH, xpath_expression)

for element in elements:
    element_text = element.text
    print("Element text:", element_text)

Answer 2

You are using find_elements method that returns a collection instead of find_element that returns single element. Collection doesn't have attribute text.

To return text from single element, you should use

driver.find_element(By.XPATH, xpath_expression).text

If you want to print all text from elements within collection, you can use loop:

for element in paragraph_elements:
    print(element.text)

Also I suggest to use more concrete xPath expression to locate your element / elements.

Something like CSS selector [role=row] .truncate a

Answer 3

You were almost there. You are looking for the (single) element with text as 63. Unique Paths II. So you instead of find_elements() you need to use find_element().

---

Solution

However to extract the visible text 63. Unique Paths II ideally you need to induce WebDriverWait for the visibility_of_element_located() and using the index [0] you can use either of the following locator strategies:

• Using CSS_SELECTOR and text attribute:

  driver.get(url='https://leetcode.com/problemset/all/?page=1&topicSlugs=array')
  elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a[href='/problems/unique-paths-ii/'][class^='h-5']")))
  print(elements[0].text)
  

• Using XPATH and get_attribute("innerHTML"):

  driver.get(url='https://leetcode.com/problemset/all/?page=1&topicSlugs=array')
  elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[starts-with(@class, 'h-5') and @href='/problems/unique-paths-ii/']")))
  print(elements[0].get_attribute('innerHTML'))
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

---

References

Link to useful documentation:

• get_attribute() method Gets the given attribute or property of the element.
• text attribute returns The text of the element.
• Difference between text and innerHTML using Selenium