How can I get text from an A tag with Selenium?

Question

I have been trying to scrape some products online, but when I try and print the title from an A tag it gives me this output:

<selenium.webdriver.remote.webelement.WebElement (session="48e7924c296324a7a5a843d9ccab36fb", element="b8871651-23af-42c6-a49a-5b93fe932653")>

Now this is my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd

PATH = "C:\Program Files (x86)\chromedriver.exe"

driver = webdriver.Chrome(PATH)
driver.get("https://egypt.souq.com")

dotd = "/html/body/div[2]/div/main/div[1]/div[1]/div/div[1]/a/img"

driver.find_element_by_xpath(dotd).click()

def get_deals():
    title_xpath = "/html/body/div[1]/div/main/div/div[4]/div[3]/div[2]/div[1]/div[1]/div/div[2]/ul/li[1]/h6/span/a"
    titles = driver.find_elements_by_xpath(title_xpath)
    for title in titles:
        print(title)

get_deals()
print("successful")

score 0 · Answer 1 · answered Jul 25 '20 at 06:58

0

The problem is that you are printing the object, which contains all the properties, instead of the text property.

So, the only thing that you need to change is, instead of using print(title), use print(title.text)

answered Jul 25 '20 at 06:58

Yoshi

172
7

score 0 · Accepted Answer · edited Nov 16 '22 at 02:46

0

This output from print()...

<selenium.webdriver.remote.webelement.WebElement (session="48e7924c296324a7a5a843d9ccab36fb", element="b8871651-23af-42c6-a49a-5b93fe932653")>

...isn’t any error, but the WebElement itself.

It seems you were close. As you were able to extract the element, to extract the text within the element, you can use either of the following Locator Strategies:

Using text attribute:

for title in titles:
    print(title.text)

Using get_attribute() attribute:

for title in titles:
    print(title.get_attribute("innerHTML"))

Reference

You can find a couple of relevant discussions in:

Difference between text and innerHTML using Selenium

edited Nov 16 '22 at 02:46

Peter Mortensen

30,738
21
105
131

answered Jul 25 '20 at 07:43

undetected Selenium

183,867
41
278
352

It **worked!** but it is only printing one title, even though the page has multiple titles – eyadali05 Jul 25 '20 at 07:48
@RazerPYOfficial Glad to be able to help you. But this sounds to be a different issue all together. Can you raise a new question with your new requirement please? – undetected Selenium Jul 25 '20 at 07:56
I posted another question – eyadali05 Jul 25 '20 at 08:38

How can I get text from an A tag with Selenium?

2 Answers2

Reference