How to parse nested XML data and extract only the userid from it?

Question

I have below xml layout from which I am suppose to extract all the "userid" value which is inside <key> </key> and load them to HashSet in Java

<?xml version="1.0" encoding="UTF-8"?>
<response>
   <plds>
      <fld>consumerid</fld>
      <fld>last_set</fld>
   </plds>
   <record>
      <data>934463448   1417753752</data>
      <key_data>
         <key>
            <name>userid</name>
            <value>934463448</value>
         </key>
      </key_data>
   </record>
   <record>
      <data>1228059948  1417753799</data>
      <key_data>
         <key>
            <name>userid</name>
            <value>1228059948</value>
         </key>
      </key_data>
   </record>
</response>

I will be getting above xml data from a url and it is possible that I can get big XML file. What is the best way to parse the above XML and extract all the "userid" and load it in the HashSet in Java?

This is what I have started -

public static Set<String> getUserList(String host, String count) {

    Set<String> usrlist = new HashSet<String>();
    String url = "urlA"; // this url will return me above XML data
    InputStream is = new URL(url).openStream();
    BufferedReader rd = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));

    // not sure what I should do here which can
    // parse my above xml and extract all the
    // userid and load it into usrlist hash set

    return usrlist;
}

UPDATE:-

This is what I have tried -

public static Set<String> getUserList() {

    Set<String> usrlist = new HashSet<String>();
    String url = "urlA"; // this url will return me above XML data
    InputStream is = new URL(url).openStream();
    BufferedReader rd = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));

    DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
    DocumentBuilder builder = factory.newDocumentBuilder();
    Document doc = builder.parse(new URL(url).openStream());

    XPathFactory xPathfactory = XPathFactory.newInstance();
    XPath xpath = xPathfactory.newXPath();
    XPathExpression expr = xpath.compile("//record/key_data/key[name='userid']/value");
    NodeList nodes = (NodeList) expr.evaluate(doc, XPathConstants.NODESET);
    for (int i = 0; i < nodes.getLength(); i++) {
        usrlist.add(nodes.item(i).getNodeValue());
    }

    return usrlist;
}

But I am not getting any user id in the usrlist object? Anything wrong I am doing here?

Could something like JAXB be of assistance? But it works in the same wat as xpath. The whole document will be in memory. — Alkis Kalogeris, Dec 05 '14 at 05:22
@alkis Can you provide an example how can I use JAXB here? I have never used JAXB before. — AKIWEB, Dec 05 '14 at 05:23
Of course. Please check my answer. Although I know consider it an overkill. — Alkis Kalogeris, Dec 05 '14 at 09:29

score 3 · Answer 1 · edited May 23 '17 at 12:05

3

If the above document is relatively small, you could load the entire document and then apply the below xpath to extract the keys in the document:

//record/key_data/key[name='userid']/value

Edit

I think you have a bug - use getTextContent() to obtain text(), not getNodeValue():

for (int i = 0; i < nodes.getLength(); i++) {
    usrlist.add( nodes.item(i).getTextContent());
}

Debug Code:

Set<String> usrlist = new HashSet<String>();
String myXml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
        "<response>\n" +
        ...
        "</response>";
InputStream is = new ByteArrayInputStream( myXml.getBytes( ) );
BufferedReader rd = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse(new InputSource(rd));

XPathFactory xPathfactory = XPathFactory.newInstance();
XPath xpath = xPathfactory.newXPath();
XPathExpression expr = xpath.compile("//record/key_data/key[name='userid']/value");
NodeList nodes = (NodeList) expr.evaluate(doc, XPathConstants.NODESET);
for (int i = 0; i < nodes.getLength(); i++) {
    usrlist.add( nodes.item(i).getTextContent());
}

return usrlist;

edited May 23 '17 at 12:05

Community

1
1

answered Dec 05 '14 at 04:53

StuartLC

104,537
17
209
285

Thanks for suggestion. It may or not be small. Sometimes it will be small and sometimes it won't be small. – AKIWEB Dec 05 '14 at 04:55
Is this correct expression to get the userid? Somehow when I am using it, I am getting null values. – AKIWEB Dec 05 '14 at 05:25
Yup, I am pretty sure - push the [transform button here](http://www.utilities-online.info/xsltransformation/?save=50e8e74b-1600-422e-9e34-0797018de3f7-xsltransformation#.VIFCyNWUcdQ). I'm guessing we have some namespaces to deal with - can you double check the input Xml is exactly as you have it? – StuartLC Dec 05 '14 at 05:31
I have updated the question with the code I am trying. Take a look whether I am doing anything wrong? – AKIWEB Dec 05 '14 at 06:03
The reason why the value is null is because need use `getTextContent` to obtain an element's `text()`, not `getNodeValue()` - I've updated. – StuartLC Dec 05 '14 at 06:42

Evgeniy Dorofeev · Answer 2 · 2014-12-05T06:36:44.517

3

StAX is an efficient way to parse large xmls

    XMLStreamReader r = XMLInputFactory.newInstance().createXMLStreamReader(is);
    while(r.hasNext()) {
        if (r.next() == XMLStreamReader.START_ELEMENT && r.getLocalName().equals("value")) {
            String value = r.getElementText();
            System.out.println(value);
        }
    }

edited Dec 05 '14 at 06:36

answered Dec 05 '14 at 06:29

Evgeniy Dorofeev

133,369
30
199
275

score 0 · Answer 3 · answered Dec 05 '14 at 08:29

I now believe that JAXB is an overkill, and that the other two answers are much better, but since you asked for an example on JAXB I'm glad to help.

You have to make the classes below, so the mapping of XML to Java objects can take place.

Response.java

import java.util.List;

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
public class Response {

    List<Plds> pldslist;
    List<Record> recordlist;

    @XmlElement(name = "plds")
    public void setPlds(List<Plds> pldslist) {
        this.pldslist = pldslist;
    }

    public List<Plds> getPlds() {
        return pldslist;
    }

    @XmlElement(name = "record")
    public void setRecord(List<Record> recordlist) {
        this.recordlist = recordlist;
    }

    public List<Record> getRecord() {
        return recordlist;
    }

}

Record.java

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
public class Record {

    String data;
    KeyData keyData;

    public String getData() {
        return data;
    }

    @XmlElement(name = "data")
    public void setData(String data) {
        this.data = data;
    }

    public KeyData getKeyData() {
        return keyData;
    }

    @XmlElement(name = "key_data")
    public void setKeyData(KeyData keyData) {
        this.keyData = keyData;
    }

}

Plds.java

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
public class Plds {

    private String fld;

    public String getFld() {
        return fld;
    }

    @XmlElement(name = "fld")
    public void setFld(String fld) {
        this.fld = fld;
    }

}

KeyData.java

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
public class KeyData {

    Key key;

    public Key getKey() {
        return key;
    }

    @XmlElement(name = "key")
    public void setKey(Key key) {
        this.key = key;
    }
}

Key.java

import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;

@XmlRootElement
public class Key {

    String name;
    String value;

    public String getName() {
        return name;
    }

    @XmlElement(name = "name")
    public void setname(String name) {
        this.name = name;
    }

    public String getValue() {
        return value;
    }

    @XmlElement(name = "value")
    public void setKey(String value) {
        this.value = value;
    }
}

Example Main.java

import java.io.File;

import javax.xml.bind.JAXBContext;
import javax.xml.bind.Unmarshaller;


public class Main {

    public static void main(String[] args) 
    throws Exception
    {
        File file = new File("PATH_HERE/file.xml");
        JAXBContext jaxbContext = JAXBContext.newInstance(Response.class);

        Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
        Response response = (Response) jaxbUnmarshaller.unmarshal(file);
        for(Record rec : response.getRecord())
            System.out.println("Name : " + rec.getKeyData().getKey().getName() + " | " + "Value : " + rec.getKeyData().getKey().getValue());
    }

}

How to parse nested XML data and extract only the userid from it?

3 Answers3