How to locate or extract texts in html file using selenium in python?

Question

How to locate or extract texts in a HTML file using Selenium in python. The text that I'm trying to get isn't an element.

<div class="datagrid row"> ==$0
 <h2 class="bottom-border block">Accepted Shipment</h2>
 <table>
  <tbody>
   <tr>
    <td>
     <input type="hidden" id="looseAcceptedPieces" value="56"> == $0
      " 56 pcs."

    <!--Modified by A-7558 for ICRD-244765 starts--> == $0
    <input type="hidden" id="acceptedWt" value> == $0
     "952 kg"

How do i locate or get that text under <input>, which is 56 pcs. and 952 kg perhaps, they are not elements.

You can read or take the entire html file as text file and then string search by "56 pcs" if "56 pcs" appears only under tag. — Aminul, Feb 28 '20 at 05:38
How about collecting text under td?, It will give you 56 pcs., 952 kg in one string may be space-separated and than you can split it. — pk786, Feb 28 '20 at 05:43

score 1 · Answer 1 · answered Feb 28 '20 at 06:31

1

You can fetch the values by using get_attribute("value") method

piece = driver.find_element_by_id('looseAcceptedPieces')
val = piece.get_attribute("value")

And

weight = driver.find_element_by_id('acceptedWt')
val2 = weight.get_attribute("value")

answered Feb 28 '20 at 06:31

Sameer Arora

4,439
3
10
20

score 0 · Answer 2 · answered Feb 28 '20 at 07:54

To extract the texts 56 pcs. and 952 kg as those are text nodes you need to induce WebDriverWait for the visibility_of_element_located() using execute_script() method and you can use either of the following xpath based Locator Strategies:

To extract 56 pcs.:

print(driver.execute_script('return arguments[0].childNodes[2].textContent;', WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='Accepted Shipment']//following::table[1]/tbody/tr/td")))).strip())

To extract 952 kg:

print(driver.execute_script('return arguments[0].lastChild.textContent;', WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='Accepted Shipment']//following::table[1]/tbody/tr/td")))).strip())

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Reference

You can find a relevent discussion in:

How to locate or extract texts in html file using selenium in python?

2 Answers2

Reference