Python selenium - get text and href

Question

Let's say I have multiple divs looking like this :

<div class="class1">
    <div class="class2">
        <div class="class3">Text1</div>
        <div class="class4">
            <a href="https://somelink"><h2>Text2</h2></a>
            <p class="class5">Text3 <span class="class6"> Text4 </span></p>
        </div>
    </div>
</div>

For each div I can get Text1, Text2, Text3, and Text4:

elements = driver.find_elements_by_xpath("//div[@class='class1']/*")
for e in elements:
    print(e.text)
    print('------------------------------------------')

But how do I additionaly get value of href?

I would like to have : https://somelink, Text1, Text2, Text3, Text4

score 1 · Answer 1 · answered Sep 26 '21 at 20:15

1

why not do this?

elements = driver.find_elements_by_xpath("//div[@class='class1']/*")
res = []
for e in elements:
    res.append(e.text)
    href = e.get_attribute('href')
    if href is not None:
        res.insert(0, href)
print(", ".join(res))

answered Sep 26 '21 at 20:15

elbud

75
6

score 1 · Accepted Answer · answered Sep 27 '21 at 05:16

Try like this:

elements = driver.find_elements_by_xpath("//div[@class='class1']/*") # this will recognize "class2" 
for e in elements:
    print(e.text)
    link = e.find_element_by_xpath(".//a").get_attribute("href") # Finds the "a" tag inside the class2. A "." at the beginning because we are finding element within elements. "//a" because "class2" is nested. 
    print('------------------------------------------')

score 0 · Answer 3 · answered Sep 26 '21 at 19:38

0

I think you'll find your answer here: Python Selenium - get href value

Basically, it will look like

driver.find_elements_by_css_selector('div.class4 > a').get_attribute('href')

answered Sep 26 '21 at 19:38

Alex

815
9
19

But if I use css selector I will not be able to get href together with Text fields. As I said, there are multiple divs with class4 so using your solution I would get a list of hrefs and second list of text fields. – psmith Sep 26 '21 at 20:05
True. Sorry, I thought your primary issue was how to get href attribute, not all the attributes at once. In that case, I would first get all the parent elements (class2, for example), and then for each such element I'd queried separately all text blocks as you did already, and separately queried a link from corresponding href attribute. An example of how to query elements inside other elements is available there: https://stackoverflow.com/questions/24795198/get-all-child-elements – Alex Sep 26 '21 at 20:21

Python selenium - get text and href

3 Answers3