How can we extract particular elements value by using xpath when element value is different for each element

Question

How can we extract ABC, XYZ by using xpath

<div id="desc" class="description">

    <span class="category">Name:</span> <span class="category-detail"><a href="/name/">Name</a></span>
    <br/>
    <span class="category">Address:</span> <span class="category-detail">ABC, XYZ</span>
    <br/>
    <span class="category">Room No:</span> <span class="category-detail">20</span>
    <br/>

I tried with

 response.xpath('//div[span="Address:"]/span/text()').extract()

but then I get [Name, ABC,XYZ, 20] but i require only ABC, XYZ.

Yes, I am using scrapy. – Pradeep Mishra Aug 21 '17 at 09:31 — Pradeep Mishra, Aug 21 '17 at 09:31

score 1 · Accepted Answer · answered Aug 21 '17 at 09:18

1

Try to use below XPath to get required output:

//div[@id="desc"]/span[.="Address:"]/following-sibling::span[1]/text()

answered Aug 21 '17 at 09:18

Andersson

51,635
17
77
129

could you please let me know why you used period in [.="Address:] – Pradeep Mishra Aug 21 '17 at 09:40
In this case it means content of `span`. You can use `text()="Address:"` as well. You can also check [this](https://stackoverflow.com/questions/38240763/xpath-difference-between-dot-and-text) to get more details – Andersson Aug 21 '17 at 09:46
any solution by using css selector ? – Pradeep Mishra Aug 21 '17 at 09:48
You can try something like `div#desc span.category-detail:nth-child(5)`, but it looks less reliable – Andersson Aug 21 '17 at 09:56
Thanks a lot with the same question: Room No: XYZ then how can we extract href value (xyz.com) from this – Pradeep Mishra Aug 21 '17 at 10:10
@PradeepMishra This [blogpost](https://blog.scrapinghub.com/2016/10/27/an-introduction-to-xpath-with-examples/) could be good introduction to XPath. – Tomáš Linhart Aug 21 '17 at 10:15
I tried with //div[@id="desc"]/span[.="Address:"]/following-sibling::span[1]/a/@href and got it :) by the way could you please tell me from where i can learn this types of complex xpath – Pradeep Mishra Aug 21 '17 at 10:17
@TomášLinhart: Thank you very much :) – Pradeep Mishra Aug 21 '17 at 10:18
@PradeepMishra, you can check [this tutorial](https://www.w3schools.com/xml/xpath_intro.asp). It can be useful for you – Andersson Aug 21 '17 at 10:20

score 0 · Answer 2 · answered Aug 21 '17 at 09:27

0

//div[@id="desc"]//span[text()='Address:']//following::span[1]/text()

Use this xpath instead.

or

WebDriver driver = new FirefoxDriver();
    driver.get("file:///C:/Users/sv/Desktop/docUpload.html");
    List<WebElement> spanList = driver.findElements(By.xpath(".//div[@id='desc']/span"));
    for (int i =0 ;i < spanList.size();i++){
        String text = spanList.get(i).getText();
        if(text.equals("ABC, XYZ")){
            System.out.println(text);
        }
    }

You can loop through the element lists.

answered Aug 21 '17 at 09:27

Sudha Velan

633
8
24

Yes, Xpath gives proper result. Thanks.. btw could you please tel me know where i can learn these types of complex xpaths – Pradeep Mishra Aug 21 '17 at 09:46
https://www.w3schools.com/xml/xpath_axes.asp https://www.guru99.com/xpath-selenium.html http://toolsqa.com/selenium-webdriver/choosing-effective-xpath/ http://www.seleniumeasy.com/selenium-tutorials/xpath-tutorial-for-selenium , Go through these sites. – Sudha Velan Aug 21 '17 at 13:35
Thank you vey much – Pradeep Mishra Aug 22 '17 at 05:25

How can we extract particular elements value by using xpath when element value is different for each element

2 Answers2