0

Well, my problem is this. I want to gather data from a web page using python and selenium, here is the html i need to get data from

 <div class="order-detail order-price">
      <div>
        <p class="item-left text--semibold">
           Subtotal
        </p>
        <p class="item-right text--semibold">$1420.00</p>
     </div>
  </div>

<The data i need is the inner text from the last "p" element, the number. I can use the Xpath for that one specific, but this is inside a ul elemten which has a lot of li elements, inside those li, you have that div code.

The problem is that the Xpath of that div changes in every li because i can have more or less div before that one specific, so i can not use Xpath, i dont have an id, a name and the class name is equal to all other div and my program can not grab one specific. Also, i need to grab one by one so i can have them order because every once in a while i have to gather another specific div and save that data in the same spot. This is my code so far

ul_principal = driver.find_element_by_xpath('/html/body/main/div[5]/section/div/div/div/div/ul/li[2]/div/div/div[1]/div[4]/ul[2]')

li_options = ul_principal.find_elements_by_tag_name('li')
for li in li_options:

     driver.implicitly_wait(100)

     li.click()

     div_Subtotal = li.find.element_by_class_name("order=detail order-price")

     if div_Subtotal is not None:
       div_Subtotal.find_element_by_class_name("item-right text--semibold").get_property('innerText')

i use li.click() to know if i can travel within the li, this works, but i can not grab the info. I can not give you the web page because its behind a password and it is private

DisappointedByUnaccountableMod
  • 6,656
  • 4
  • 18
  • 22
Nicolas Dascanio
  • 1

2 Answers2

0

Try something like BS4.find(class_ = "order-detail order-price").find("p", {"class": "item-right"}).text

AlanWik
  • 326
  • 1
  • 10
0

To extract and print the text $1420.00 you can use either of the following Locator Strategies:

  • Using css_selector and get_attribute():

    print(driver.find_element_by_css_selector("ul li div.order-detail.order-price p.item-right.text--semibold").get_attribute("innerHTML"))

  • Using xpath and text attribute:

    print(driver.find_element_by_xpath("//ul//li//div[@class='order-detail order-price']//p[@class='item-right text--semibold']").text)

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR and text attribute:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "ul li div.order-detail.order-price p.item-right.text--semibold"))).text)

  • Using XPATH and get_attribute():

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//ul//li//div[@class='order-detail order-price']//p[@class='item-right text--semibold']"))).get_attribute("innerHTML"))

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

References

Link to useful documentation:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352