Xpath nested spans

Question

I have the following html:

<div id="aod-price-1" class="a-section a-spacing-none a-padding-none">
    <span class="a-price" data-a-size="l" data-a-color="base">
        <span class="a-offscreen">$79.58</span>
        <span aria-hidden="true">
            <span class="a-price-symbol">$</span>
            <span class="a-price-whole">
                "79"
                <span class="a-price-decimal">.</span>
            </span>
            <span class="a-price-fraction">58</span>
        </span>
    </span>
</div>

I am trying to extract the $79.58.

I used:

priceFound = WebDriverWait(browser,10).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='a-price']")))

This seems to work but not quite the way I intended:

It returns:

$79
58

2 separate lines, no decimal

I am trying to extract the intact text string: $79.58

I even tried:

priceFound = WebDriverWait(browser,10).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='a-offscreen']")))

and

priceFound = WebDriverWait(browser,10).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='a-price-whole']")))

Those 2 did not work.

UPDATE BASED ON SUGGESTIONS SO FAR:

Please note that priceFound is a list and there are several blocks like the one above in the actual html (many prices).

<div id="aod-price-1" ... </div>
<div id="aod-price-2" ... </div>
<div id="aod-price-3" ... </div>
<div id="aod-price-4" ... </div>

I just posted one block for clarity and is why I chose a list.

priceFound = WebDriverWait(browser, 10).until(EC.presence_of_all_elements_located((By.XPATH, "//span[@class='a-price']/span[@class='a-offscreen']")))

for price in priceFound:
    print(price.text)

This returned: several blank lines (blank carriage returns to be specific)

I wonder if I need a .text reference somewhere in XPath?

UPDATE 2:

I use the following to click on the see all buying choices button. It does work. Then introduce the slight wait to wait for prices to populate.

  Expand_button_Element = browser.find_element_by_id("buybox-see-all-buying-choices")
  Expand_button_Element.click()

UPDATE 3:

 wait = WebDriverWait(browser, 10);
        # wait for panel to be visible
     wait.until(EC.visibility_of_element_located((By.ID, "aod-container")))
        # this wait is probably no longer needed but left in to be safe
     priceFound = wait.until(EC.visibility_of_all_elements_located((By.XPATH,"//span[@class='a-price']")))
     for price in priceFound:
        print(price.text)

`

Produces (as an example): $77 23 $77 24 $79 59 $79 94 $78 95 $83 94 $79 99 $79 95 $79 99 $89 00

But when I try the code suggestion below:

browser.find_element_by_id("buybox-see-all-buying-choices").click()
     wait = WebDriverWait(browser, 10);
     # wait for panel to be visible
     wait.until(EC.visibility_of_element_located((By.ID, "aod-container")))
     # this wait is probably no longer needed but left in to be safe
     priceFound = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "span.a-price > span.a-offscreen")))
     for price in priceFound:
        print(price.text)

I get the following error:

 priceFound =   
wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "span.a-
price > span.a-offscreen")))
File "/home/codingArea/.local/lib/python3.8/site-packages/selenium/webdriver
/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

This seems like a similar problem:

How to get text from nested span tag in selenium

I thought this would work for sure:

priceFound = wait.until(EC.visibility_of_all_elements_located((By.XPATH,"//span[@class='a-price']/span[@class='a-offscreen']")))

but it timed out which makes no sense to me. I made sure I was running the lastest selenium.

UPDATE 4: I used the following and it did not error but it produced empty lines (like 10 carriage returns).

priceFound = browser.find_elements_by_css_selector('span.a-offscreen')

     for price in priceFound:
        print(price.text)

Can you post a link to the page? – JeffC Apr 06 '21 at 16:28 — JeffC, Apr 06 '21 at 16:28
Example page: https://www.amazon.com/dp/B08WJJNT2R – NeilS Apr 06 '21 at 17:28 — NeilS, Apr 06 '21 at 17:28

JeffC · Answer 1 · 2021-04-06T17:55:35.950

0

You've tried a number of approaches but you didn't post what the results of each were. So to baseline, let's start with something simple and see if it solves the issue. If it doesn't, we can work from there.

Let's use a simple CSS selector and add a wait for visibility. (NOTE: you are using presence but that just means that the element is in the DOM, not that it's visible)

# this code starts after clicking link to open product panel with pricing
browser.find_element_by_id("buybox-see-all-buying-choices").click()
wait = WebDriverWait(browser, 10);
# wait for panel to be visible
wait.until(EC.visibility_of_element_located((By.ID, "aod-container")))
# this wait is probably no longer needed but left in to be safe
priceFound = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "span.a-price > span.a-offscreen")))
for price in priceFound:
    print(price.text)

edited Apr 06 '21 at 17:55

answered Apr 06 '21 at 16:56

JeffC

22,180
5
32
55

The text does seem to exist because: WebDriverWait(browser, 10).until(EC.presence_of_all_elements_located((By.XPATH, "//div[@id='all-offers-display']"))) works. It just pulls the data in a format I don't want (i.e. split with a carriage return which I assume is from further in the DOM) – NeilS Apr 06 '21 at 17:53
Now that you've provided a link to the page, I've updated the code. You forgot to mention in your question that you clicked a link to open a panel that contains the prices. That's likely where the issue is coming from so I added a wait for the panel. See if this works. – JeffC Apr 06 '21 at 17:53
The locator in your comment is WAY too high up to use... you are going to get all text in the entire panel. I think you are confusing what you've tried. – JeffC Apr 06 '21 at 17:59

Xpath nested spans

1 Answers1