How to correct ValidatorException in Java SSL REST client

Question

I have a RESTful client written in Java for a web service that uses SSL. The application has been working fine for several weeks. All of a sudden all attempts to transact with the web service result in the following exception:

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1699)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:893)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1165)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1149)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:166)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1172)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:234)
at com.MSCA.ESP.WebApp.ABSManager.getFacilityMetrics(ABSManager.java:1000)
....

The information I've found regarding similar difficulties, like this one, seem to indicate that I need to do something in my client environment with a certificate provided by/for the web service. However, I continue to be able to execute the same transactions with RESTclient in Firefox that are failing in my Java client, and to my knowledge I've never provided RESTclient with any sort of certificate related to that service.

The back story of this application is that the web service was originally based on the SOAP protocol, and was recently changed over to REST, which adds another level of complexity and confusion, at least from my perspective, though maybe that's a clue to someone else as to what my problem might be. On top of that, the only thing I remember about dealing with SSL when I first developed the application a couple years ago is that I had to add the following code to it:

static
{
    Security.addProvider( new com.sun.net.ssl.internal.ssl.Provider() );
    System.setProperty( "java.protocol.handler.pkgs", "com.sun.net.ssl.internal.www.protocol" );
    System.setProperty( "javax.net.ssl.trustStore", "./Resources/client.keystore" );
}

The client.keystore file was originally provided with the sample client code for the SOAP interface. I left this code and the client.keystore file in place once the service was switched over to REST, as it didn't seem to cause a problem, even though client.keystore doesn't appear to be provided to developers writing RESTful clients strictly for the new service. I submitted a support request to the web service provider today asking about possibly needing an update to client.keystore. My difficulty didn't seem to ring a bell with them, so I'm thinking that perhaps I'm barking up the wrong tree there.

I don't know if I've provided enough information here for anyone to give me a complete answer, but at this point I'd be happy with merely a particular vector to pursue. So far I seem to be running in circles. Finally, I don't know if this is significant, but my application runs under Glassfish 3.1.1 on Windows Server 2003 Standard Edition.

Answer 1

After a virtual crash course in web security, I finally surmised that the root certificates in my application's environment needed to be updated. I found a utility called InstallCert.java that will determine the root certificates needed by a particular host and write them to a file. I used it to create such a file, also called a truststore, and renamed it to "cacerts.jks", which is the filename used by Glassfish (3.1.1) for the truststore. I copied it over both copies of cacerts.jks in the Glassfish environment (found in /glassfish3/glassfish/lib/templates and /glassfish3/glassfish/domains/domain1/config), restarted the Glassfish instance and all was well.

A couple blog entries that helped me figure this out were Updating the root certificates for Java and unable to find valid certification path to requested target. A couple mysteries/questions remain from my perspective:

1. The truststore generated by InstallCert.java appears to include many root certificates that are not used by the host I needed an updated root certificate for. Where did these come from? Are they the certificates already found in my current Java installation? If so, where in the InstallCert.java code are they written to the output file?

2. How can one foresee and/or avoid this problem? Since root certificates seem to be distributed primarily by way of development tools and application servers, and for the most part are out of sight and out of mind, is there any way to be notified that one that your application depends on is becoming obsolete? Or are we expected to regularly update the development tools and application servers used in production environments to make it less likely to be stung by such a situation?