How do I get Java/HtmlUnit to accept an old or bad website certificate?

Question

If you go to https://eagletw.mohavecounty.us/treasurer/treasurerweb/search.jsp using Firefox on Linux, you will see that you can browse the website just fine.

But when I compile and run the following program:

import com.gargoylesoftware.htmlunit.html.*;
import com.gargoylesoftware.htmlunit.javascript.*;
import java.io.*;

public class BadWebsiteCertificate {
    public static void BadWebsiteCertificate () {
        try (final WebClient webClient = new WebClient()) {
            System.getProperties().put("org.apache.commons.logging.simplelog.defaultlog", "fatal");
            webClient.getOptions().setThrowExceptionOnScriptError(false);
            webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);

            webClient.getOptions().setCssEnabled(false);
            webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
            webClient.setCssErrorHandler(new SilentCssErrorHandler());
            webClient.setAjaxController(new NicelyResynchronizingAjaxController());
            HtmlPage page = webClient.getPage("https://eagletw.mohavecounty.us/treasurer/treasurerweb/search.jsp");
            webClient.waitForBackgroundJavaScriptStartingBefore(10000);
            page = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();
            webClient.getOptions().setThrowExceptionOnScriptError(false);
            webClient.setJavaScriptErrorListener(new SilentJavaScriptErrorListener());
            HtmlTable grdTaxHistory = (HtmlTable) page.getElementById("grdTaxHistory");
            HtmlTableDataCell cpCell = (HtmlTableDataCell) grdTaxHistory.getCellAt(4,6);
            ((HtmlAnchor) cpCell.getFirstChild().getNextSibling()).click();

            webClient.waitForBackgroundJavaScriptStartingBefore(1_000);
            page = (HtmlPage) page.getEnclosingWindow().getEnclosedPage();
        }

        catch (Exception e) {
            System.out.println("Error: "+ e);
        }

    }

    public static void main(String[] args) {
        File file = new File("validParcelIDs.txt");
        BadWebsiteCertificate();
    }

}

using the following commands:

javac -classpath ".:/opt/htmlunit_2.69.0/*" BadWebsiteCertificate.java
java -classpath ".:/opt/htmlunit_2.69.0/*" BadWebsiteCertificate

I get the following runtime error message:

Error: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I tried the following solution proposed at Resolving javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed Error?

echo -n | openssl s_client -connect eagletw.mohavecounty.us:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/eagletw.mohavecounty.us.crt
sudo keytool -import -v -trustcacerts -alias eagletw.mohavecounty.us -file ~/eagletw.mohavecounty.us.crt -keystore /usr/lib/jvm/java-11-openjdk-amd64/lib/security/cacerts -keypass changeit -storepass changeit

But that didn't fix the problem. I am still getting the same runtime error message.

Any ideas of what else I can try?

[That server is broken](https://www.ssllabs.com/ssltest/analyze.html?d=eagletw.mohavecounty.us) -- it does not provide the intermediate aka chain cert as required by standards but instead a useless second copy of the leaf cert. Browsers like Firefox can _sometimes_ work-around this, probably using AIA, but Java doesn't. ... — dave_thompson_085, Jan 22 '23 at 20:43
... That said putting it in the default truststore should work (although it's not very secure) IF your code (i.e. htmlunit) uses the default truststore, which I don't know; however openjdk on most Linux distros, if that is what you are using, links `cacerts` to a package-supplied file in something like /etc/pki so any writes to it may be replaced automatically by system data, then you need to use a different file and `-Djavax.net.ssl.trustStore` or equivalent. — dave_thompson_085, Jan 22 '23 at 20:44

score 0 · Answer 1 · answered Jan 22 '23 at 20:25

For some reason your Java runtime environment does not trust that certificate. There are two things you can try:

add that website's certificate to your truststore manually
upgrade to the latest JVM and see if the truststore in that version already solves your problem

I'd go first for upgrade, if that does not work check how to add certificates to Java's truststore. See also

score 0 · Answer 2 · answered Jan 23 '23 at 09:02

0

You can disable the SSL check by

webClient.getOptions().setUseInsecureSSL(true);

This might also help if you use a proxy like Charles Web Proxy to inspect the traffic.

answered Jan 23 '23 at 09:02

RBRi

2,704
2
11
14

How do I get Java/HtmlUnit to accept an old or bad website certificate?

2 Answers2