XpathException in Java MAVEN example Expression uses unbound namespace prefix

Question

After researching on google I have not find a working solution for this. The 'MAVEN by Example' ebook uses the Yahoo weather example. Unfortunately it looks like Yahoo changed their interface. I tried to adapt the java code for this, but get this annoying exception:

        exec-maven-plugin:1.5.0:java
     Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java
            Caused by: org.dom4j.XPathException: 
Exception occurred evaluting XPath: /query/results/channel/yweather:location/@city. 
Exception: XPath expression uses unbound namespace prefix yweather

The xml line itself is:

<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng" yahoo:count="1" yahoo:created="2017-02-13T10:57:34Z" yahoo:lang="en-US">
<results>
    <channel>
    ...
        <yweather:location xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" city="Theale" country="United Kingdom" region=" England"/>

The entire XML can be generated from :

https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%3D91731537

My code (as per the 'MAVEN By Example' ebook, xpath and url modified for the changed Yahoo):

    public Weather parse(InputStream inputStream) throws Exception {
    Weather weather = new Weather();

    SAXReader xmlReader = createXmlReader();
    Document doc = xmlReader.read( inputStream );
    weather.setCity(doc.valueOf  ("//yweather:location/@city") );
   // and several more, such as setCountry, setTemp 
}

(I'm not an xpath expert, so I tried

/query/results/channel/item/yweather:location/@city

as well, just in case, with the same result.

xmlReader:

public InputStream retrieve(String woeid) throws Exception {
        String url = "https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20weather.forecast%20where%20woeid%3D"+woeid; // eg 91731537
        URLConnection conn = new URL(url).openConnection();
        return conn.getInputStream();
    }

and the weather class is just a set of getters and setters

When I try this in this XML tester, it works just fine, but that may be the effect of XPATH-v2 vs Java's v1.

Answer 1

When you evaluate your XPath //yweather:location/@city, the XPath processor has no knowledge of which namespace the yweather prefix is bound to. You'll need to provide that information. Now, you might think "the info is right there in the document!" and you'd be right. But prefixes are just a sort of stand-in (like a variable) for the actual namespace. A namespace can be bound to any prefix you like that follows the prefix naming rules, and can be bound to multiple prefixes as well. Just like the variable name in Java referring to an object is of itself of no importance, and multiple variables could refer to the same object.

For example, if you used XPath //yw:location/@city with the prefix yw bound to namespace http://xml.weather.yahoo.com/ns/rss/1.0, it'd still work the same.

I suggest you use class org.dom4j.xpath.DefaultXPath instead of calling valueOf. Create an instance of it and initialize the namespace context. There's a method setNamespaceURIs that takes a Map from prefixes to namespaces and lets you make the bindings. Bind the above weather namespace (the actual URI) to some prefix of your choosing (may be yweather, but can be anything else you want to use in your actual XPath expression) and then use the instance to evaluate it over the document.

Here's an answer I gave to some question that goes more in-depth about what namespaces and their prefixes really are: https://stackoverflow.com/a/8231272/630136

EDIT: the online XPath tester you used probably does some behind-the-scenes magic to extract the namespaces and their prefixes from the given document and bind those in the XPath processor.

If you look at their sample XML and adjust it like this...

<root xmlns:foo="http://www.foo.org/" xmlns:bar="http://www.bar.org">
    <actors>
        <actor id="1">Christian Bale</actor>
        <actor id="2">Liam Neeson</actor>
        <actor id="3">Michael Caine</actor>
    </actors>
    <foo:singers xmlns:test="http://www.foo.org/">
        <test:singer id="4">Tom Waits</test:singer>
        <foo:singer id="5">B.B. King</foo:singer>
        <foo:singer id="6">Ray Charles</foo:singer>
    </foo:singers>
</root>

the XML is semantically equivalent, because the test prefix is bound to the same namespace as foo. The XPath //foo:singer/@id still returns all the right results, so the tool is smart about it. However, it doesn't know what to do with XML...

<root xmlns:foo="http://www.foo.org/" xmlns:bar="http://www.bar.org">
    <actors>
        <foo:actor id="1">Christian Bale</foo:actor>
        <actor id="2">Liam Neeson</actor>
        <actor id="3">Michael Caine</actor>
    </actors>
    <foo:singers xmlns:test="http://www.foo.org/" xmlns:foo="http://www.bar.org">
        <test:singer id="4">Tom Waits</test:singer>
        <foo:singer id="5">B.B. King</foo:singer>
        <foo:singer id="6">Ray Charles</foo:singer>
    </foo:singers>
</root>

and XPath //foo:*/@id. The prefix foo is bound to a different namespace in the singers element scope, and now it only returns the ids 5 and 6. Contrast it with this XPath, that doesn't use a prefix but the namespace-uri() function: //*[namespace-uri()='http://www.foo.org/']/@id

That last one returns ids 1 and 4, as expected.

Answer 2

I found the error, it's my unfamiliarity with namespaces. The 'createXmlReader()' used in my example above is a method that sets the correct namespace, except that I forgot to change it after Yahoo changed the xml. Careful re-reading the Maven-by-example documentation, the generated error, and comparing with the detailed answer given here, it suddenly clicked. The updated code (for the benefit of anyone trying the same example):

private SAXReader createXmlReader() {
    Map<String,String> uris = new HashMap<String,String>();
    uris.put( "yweather", "http://xml.weather.yahoo.com/ns/rss/1.0" );
    DocumentFactory factory = new DocumentFactory();
    factory.setXPathNamespaceURIs( uris );
    SAXReader xmlReader = new SAXReader();
    xmlReader.setDocumentFactory( factory );
    return xmlReader;
}

The only change is in the line 'uris.put()' Originally the namespace was "y", now it is "yweather".