Selenium Python, extract text from node and ALL child nodes

Question

I have the opposite problem described here. I can't get the text more than one layer deep.

HTML is structured in the following manner:

    <span class="data">
        <p>This text is extracted just fine.</p>
        <p>And so is this.</p>
        <p>
            And this.
            <div>
                <p>But this text is not extracted.</p>
            </div>
        </p>
        <div>
            <p>And neither is this.</p>
        </div>
    </span>

My Python code looks something like this:

    el.find_element_by_xpath(".//span[contains(@class, 'data')]").text

score 0 · Answer 1 · answered Jul 21 '19 at 20:30

Try the same with child elements:

print(el.find_element_by_xpath(".//span[contains(@class, 'data')]").text)
print(el.find_element_by_xpath(".//span[contains(@class, 'data')]/div").text)
print(el.find_element_by_xpath(".//span[contains(@class, 'data')]/p").text)

score 0 · Answer 2 · answered Jul 22 '19 at 03:33

Not sure what's the referred el in your original post. But able to get all the text using the below.

 driver.find_element_by_xpath("//span[@class='data']").text

Output:

'This text is extracted just fine.\nAnd so is this.\nAnd this.\nBut this text is not extracted.\nAnd neither is this.'

score 0 · Answer 3 · answered Jul 22 '19 at 12:40

Instead of relying on WebElement.text property consider querying innerText property
Consider using Explicit Wait as it will make your test more robust and reliable in case if the element you're looking for is loaded by i.e. AJAX call

Assuming all above:

print(WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//span[@class='data']"))).get_attribute("innerText"))

Demo:

Selenium Python, extract text from node and ALL child nodes

3 Answers3