How to extract the text 121.6 from the text node within the span tag using Selenium and Python

Question

For the following element on the web

<span title="点赞数14332" class="like"><!----><!----><!----><!----><!----><i class="van-icon-videodetails_like" style="color:;"></i>1.4万
    </span>

<span title="number" class="like">
   <!---->
   <!---->
   <!---->
   <!---->
   <!---->
   <i class="van-icon-videodetails_like" style="color:;"></i>
"121.6"
</span>

What I want to get is the number "121.6". I tried,

likes = driver.find_elements_by_xpath('//span[@class="like"]')[0].text

It returns me "--", and nothing else.

I also tried to copy the X-path from the inspection,

likes = driver.find_elements_by_xpath('//*[@id="arc_toolbar_report"]/div[1]/span[1]/text()')

But selenium returns me:

selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: The result of the xpath expression "//*[@id="arc_toolbar_report"]/div[1]/span[1]/text()" is: [object Text]. It should be an element.

What should I do to get the number "121.6"?

undetected Selenium · Answer 1 · 2020-05-28T13:20:15.880

0

To extract the number of Likes on the video i.e. the current text as 1.5万 as the text is within a Text Node you need to induce WebDriverWait for the visibility_of_element_located() and execute_script() method and you can use either of the following Locator Strategies:

Using XPATH:

print(driver.execute_script('return arguments[0].lastChild.textContent;', WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[@class='like' and starts-with(@title, '点赞数')][not(contains(.,'--'))]")))).strip())

Using CSS_SELECTOR:

print(driver.execute_script('return arguments[0].lastChild.textContent;', WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "span.like[title^='点赞数']")))).strip())

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Reference

You can find a couple of relevant discussions in:

edited May 28 '20 at 13:20

answered May 28 '20 at 06:51

undetected Selenium

183,867
41
278
352

It still return's me '--' instead of the number – Xiangbo Gao May 28 '20 at 07:11
@XiangboGao Did you try both the _xpath_ and _Css_ – undetected Selenium May 28 '20 at 07:14
Yes I did. For CSS, since title is not always 'number'. I used ```span.like``` instead. That is not the problem, right? – Xiangbo Gao May 28 '20 at 07:17
It is a chinese website. https://www.bilibili.com/video/BV18K4y1t7De and the full X-path is this, if you want to look it up /html/body/div[3]/div/div[1]/div[3]/div[1]/span[1]/text() – Xiangbo Gao May 28 '20 at 07:20
And please look it up. I am very new in selenium and I am struggling in this for hours – Xiangbo Gao May 28 '20 at 07:24
Oh, if you are still working on this problem. Just to mention that the number I want to get is dynamic. It changes over time. May be that is why we cannot get it? on another X-path, there is a number which is inside the
, but I still got '--', instead of the actually number.
– Xiangbo Gao May 28 '20 at 07:33
@XiangboGao Instead of handcrafted HTML update the question with text based actual HTML. – undetected Selenium May 28 '20 at 07:34
Now, it has been updated. It that what you are looking for? – Xiangbo Gao May 28 '20 at 07:44
@XiangboGao Checkout the updated answer and let me know the status. – undetected Selenium May 28 '20 at 13:20

rhn27 · Answer 2 · 2020-05-28T10:20:52.830

I've tested your provided xpath and your link to bilibili.com and was able to get the number of likes for that video. As already mentioned in DebanjanB's answer you will have to use WebDriverWait.

Your code returned '--' because the amount of likes were still dynamically being loaded in the background. Below you will find a code snippet that waits for the '--' to be replaced by an actual number

WebDriverWait(driver, 120).until(
    EC.visibility_of_element_located((By.XPATH,'/html/body/div[3]/div/div[1]/div[3]/div[1]/span[1] \
    [not(contains(text(),"--"))]')
))

likes = driver.find_element_by_xpath("/html/body/div[3]/div/div[1]/div[3]/div[1]/span[1]").text
print(likes)

which returns 1.4万

The

[not(contains(text(),"--"))]

basically tells the driver to wait until the selected node does not contain the '--' string.

EDIT:

from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

These are the imports which you will need.

How to extract the text 121.6 from the text node within the span tag using Selenium and Python

2 Answers2

Reference