Python and Selenium - get text excluding child node's text

Question

Using Python 3.

Supposing:

<whatever>
  text
  <subchild>
    other
  </subchild>
</whatever>

If I do:

elem = driver.find_element_by_xpath("//whatever")

elem.text contains "text other"

If I do:

elem = driver.find_element_by_xpath("//whatever/text()[normalize-space()]")

elem is not Webelement.

How my I proceed to grab only "text" (and not "other")?

Id est: grab only text in direct node, not the child nodes.

UPDATE:

Original HTML is:

<div class="border-ashes the-code text-center">
VIVEGRPN
  <span class="cursor"></span>
  <button class="btn btn-ashes zclip" data-clipboard-target=".the-code" data-coupon-code="VklWRUdSUE4=">
  <span class="r">Hen, la.</span>
</div>

do the tags have ids or classes? Or are they just plain html tags? — Oceanic_Panda, Jul 21 '17 at 13:03
They have ids and classes. I updated the question with the original HTML. — Álvaro N. Franz, Jul 21 '17 at 13:04
so then if div has a class of "border-ashes the-code text-center" what text does this return: driver.find_element_by_xpath("//div[@class='border-ashes the-code text-center']") — Oceanic_Panda, Jul 21 '17 at 13:06
Does this answer your question? [How to get text of an element in Selenium WebDriver, without including child element text?](https://stackoverflow.com/questions/12325454/how-to-get-text-of-an-element-in-selenium-webdriver-without-including-child-ele) — Pikamander2, Apr 15 '20 at 04:01

score 8 · Answer 1 · edited Jun 23 '23 at 14:31

Bear in mind that the replacement approach mentioned by @Guy doesn't work for many structures.

For instance, having this structure:

<div>
    Hello World
    <b>e</b>
</div>

The parent text would be Hello World e, the child text would be e, and the replacement would result in Hllo World instead of Hello World.

A safe solution

To get the own text of an element in a safe manner, you have to iterate over the children of the node, and concat the text nodes. Since you can't do that in pure Selenium, you have to execute JS code.

OWN_TEXT_SCRIPT = "if(arguments[0].hasChildNodes()){var r='';var C=arguments[0].childNodes;for(var n=0;n<C.length;n++){if(C[n].nodeType==Node.TEXT_NODE){r+=' '+C[n].nodeValue}}return r.trim()}else{return arguments[0].innerText}"
parent_text = driver.execute_script(OWN_TEXT_SCRIPT, elem)

The script is a minified version of this simple function:

if (arguments[0].hasChildNodes()) {
    var res = '';
    var children = arguments[0].childNodes;
    for (var n = 0; n < children.length; n++) {
        if (children[n].nodeType == Node.TEXT_NODE) {
            res += ' ' + children[n].nodeValue;
        }
    }
    return res.trim()
}
else {
    return arguments[0].innerText
}

score 5 · Answer 2 · answered Jul 21 '17 at 13:12

I had similar problem recently, where selenium always gave me all the text inside the element including the spans. I ended up splitting the string with newline "\n". for e.g.

all_text = driver.find_element_by_xpath(xpath).text
req_text = str.split(str(all_text ), "\n")[0]

score 5 · Accepted Answer · answered Jul 21 '17 at 13:22

5

You can remove the child node text from the all text

all_text = driver.find_element_by_xpath("//whatever").text
child_text = driver.find_element_by_xpath("//subchild").text

parent_text = all_text.replace(child_text, '')

answered Jul 21 '17 at 13:22

Guy

46,488
10
44
88

12

this could cause problems in case they both have the same text – Alon Gouldman Mar 15 '19 at 08:51

score 0 · Answer 4 · edited Jun 09 '21 at 11:37

0

You can firstly extract the outerHTML from the element, then build the soup with BeautifulSoup, and remove any element you want.

Small example:

el = driver.find_element_by_css_selector('whatever')
outerHTML = el.get_attribute('outerHTML')
soup = BeautifulSoup(outerHTML)
inner_elem = soup.select('subchild')[0].extract()
text_inner_elem = inner_elem.text
text_outer_elem = soup.text

edited Jun 09 '21 at 11:37

Jack Miller

6,843
3
48
66

answered May 21 '20 at 09:02

magic_of_gnu

1

Python and Selenium - get text excluding child node's text

4 Answers4

Linked