Https request with mutual authentication passes with curl but fails with java

Question

Someone on github asked me a question regarding my library. This library provides some factory classes to easily create a sslcontext. I make sure not the share the details of the library and just share plain java code and the additional library which I used. I tried to debug it and also tried reformatting of the certificates but it does not work. The exception which I got is:

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The java trustmanager is basically saying I received the certificate of the server and I just checked in your list of trusted certificates and I couldn't find a matching certificate. So I don't trust this server and therefor I am stopping this request. When I debugged it I could understand why the trustmanager throw an exception because there was no matching certificate at all which was the same as the server. However when I use the same files with curl it works, so I was not quite sure if there is an issue with the certificates and the key material, or maybe java ssl implementation has some limitations or maybe I am not configuring it correctly... I hope someone can explain me the difference in behaviour of why it fails with java and passes with curl.

The private key content of the client:

The certificate chain content of the client:

The trusted certificate, ca

So the following curl command will successfully execute and gives me the correct response body:

curl --cacert ca.pem --key client-key.pem --cert client-cert.pem https://prod.idrix.eu/secure/

To load these files in Java I use Bouncy Castle, below is the full snippet of parsing it, loading it into TrustManager, KeyManager, creating the SSLContext and executing the request:

import org.bouncycastle.asn1.pkcs.PrivateKeyInfo;
import org.bouncycastle.cert.X509CertificateHolder;
import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.openssl.PEMKeyPair;
import org.bouncycastle.openssl.PEMParser;
import org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter;

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.io.StringReader;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.security.KeyStore;
import java.security.PrivateKey;
import java.security.cert.Certificate;
import java.security.cert.X509Certificate;
import java.util.Objects;
import java.util.stream.Collectors;

public class App {

    private static final BouncyCastleProvider BOUNCY_CASTLE_PROVIDER = new BouncyCastleProvider();
    private static final JcaPEMKeyConverter KEY_CONVERTER = new JcaPEMKeyConverter().setProvider(BOUNCY_CASTLE_PROVIDER);
    private static final JcaX509CertificateConverter CERTIFICATE_CONVERTER = new JcaX509CertificateConverter().setProvider(BOUNCY_CASTLE_PROVIDER);

    public static void main(String[] args) throws Exception {
        String clientKeyContent = App.getContent(App.class.getClassLoader().getResourceAsStream("client-key.pem"));
        String clientCertificateChainContent = App.getContent(App.class.getClassLoader().getResourceAsStream("client-cert.pem"));
        String caCertificateContent = App.getContent(App.class.getClassLoader().getResourceAsStream("ca.pem"));

        Reader reader = new StringReader(clientKeyContent);
        PEMParser pemParser = new PEMParser(reader);
        Object object = pemParser.readObject();

        PrivateKeyInfo privateKeyInfo = ((PEMKeyPair) object).getPrivateKeyInfo();
        PrivateKey clientKey = KEY_CONVERTER.getPrivateKey(privateKeyInfo);
        reader.close();
        pemParser.close();

        reader = new StringReader(clientCertificateChainContent);
        pemParser = new PEMParser(reader);

        object = pemParser.readObject();
        X509Certificate clientCertificateChain = CERTIFICATE_CONVERTER.getCertificate((X509CertificateHolder) object);
        reader.close();
        pemParser.close();

        reader = new StringReader(caCertificateContent);
        pemParser = new PEMParser(reader);

        object = pemParser.readObject();
        X509Certificate caCertificate = CERTIFICATE_CONVERTER.getCertificate((X509CertificateHolder) object);
        reader.close();
        pemParser.close();

        KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        keyStore.load(null, "password".toCharArray());
        keyStore.setKeyEntry("client", clientKey, "".toCharArray(), new Certificate[]{clientCertificateChain});
        KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
        keyManagerFactory.init(keyStore, "".toCharArray());

        KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
        trustStore.load(null, "password".toCharArray());
        trustStore.setCertificateEntry("ca", caCertificate);
        TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        trustManagerFactory.init(trustStore);

        SSLContext sslContext = SSLContext.getInstance("TLS");
        sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), null);

        HttpClient httpClient = HttpClient.newBuilder()
                .sslContext(sslContext)
                .build();

        HttpRequest request = HttpRequest.newBuilder()
                .GET()
                .uri(URI.create("https://prod.idrix.eu/secure/"))
                .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
        System.out.println(response.body());
    }

    public static String getContent(InputStream inputStream) throws IOException {
        try (InputStreamReader inputStreamReader = new InputStreamReader(Objects.requireNonNull(inputStream), StandardCharsets.UTF_8);
             BufferedReader bufferedReader = new BufferedReader(inputStreamReader)) {

            return bufferedReader.lines()
                    .collect(Collectors.joining(System.lineSeparator()));
        }
    }

}

I am using Java 11 and Bouncy Castle, see below for the specific artifact-id:

<dependency>
    <groupId>org.bouncycastle</groupId>
    <artifactId>bcpkix-jdk15on</artifactId>
    <version>1.68</version>
</dependency>

So for me it is not clear why the curl command works and the setup in java fails. Did I configured something wrong, any help would be great! I tried to provide as much as details possible. The private key is a test key so it won't harm to share it here.

if I remember correctly, there is a property that need to be modified in the java.security file that you can find inside your jdk folder at the path \jre\lib\security — Zartof, Jan 09 '21 at 22:51
Do you mean `Unlimited cryptography` check within the `jre/lib/security/policy/`? — Hakan54, Jan 09 '21 at 23:02
No, the file i'm talking about it's in the same folder of the default path of the java Truststore. Is your keystore path property correctly defined in configuration? Have you tried to add the certificate to the keystore like explained here https://stackoverflow.com/questions/4325263/how-to-import-a-cer-certificate-into-a-java-keystore ? If it's a web application did you set the environment property to the keystore path file? **This also may be useful (https://stackoverflow.com/questions/5871279/java-ssl-and-cert-keystore) — Zartof, Jan 10 '21 at 14:16

score 1 · Accepted Answer · answered Jan 23 '21 at 10:16

1

I discovered what the issue was and so I wanted to share with others who have this in common.

It looks like on mac os x curl is by default using the system trusted certificates and on java side this is not the case. I am not sure what the behaviour of curl is with other operating systems. https://prod.idrix.eu/secure/ has a certificate which is signed by DST Root CA X3 I disabled this and rerun the exact same setup as described on my initial question and the curl https request fails, just like it also failed on java side.

I disabled it from keychain as shown here:

answered Jan 23 '21 at 10:16

Hakan54

3,121
1
23
37

How did you know which cert was leading to this problem? Cause I have a similar problem but vice versa. I am trying to POST to a server via Java app and I get an 403 all the time. However when using Postman/curl/SoapUI it works fine, so I have the assumption that I might need to add a cert or the like? – doct0re May 25 '21 at 19:07
Well on java side I don't use the default truststore, I prefer to have a custom truststore which only contains the specific trusted certificate for my use case. Having this setup I could easily see that the certificate was needed on java side to be added in the truststore to get a good ssl handshake. I found it strange that I didn't need to specify the server certificate when using curl, so my assumption was that curl is by default using some trusted root ca. In that way I started to further analyse it and I found that indeed that curl was using default list of trusted certificates – Hakan54 May 25 '21 at 21:40
So do I understand correct that you have been adding DST Root CA X3 cert to your truststore in order to be able to have a successful handshake via Java app? – doct0re May 26 '21 at 19:39
Yes that's correct. However you may need to add a different certificate if the server you are calling has a different root ca. Which URL are you trying to access, is it public? – Hakan54 May 26 '21 at 19:46
Yes it is public and it uses DST Root CA X3 as root ca as well :-) It´s: https://marketplace.syncier.com/ – doct0re May 27 '21 at 18:13
I have a similar issue, that´s why I was checking this post: https://stackoverflow.com/questions/67876974/mutual-tls-works-with-postman-or-soapui-but-doesnt-work-with-java-springboot maybe you have an idea why this is happening? any help would be highly appreciated. – doct0re Jun 10 '21 at 08:27
I will try to help, I have dropped a comment on your question – Hakan54 Jun 10 '21 at 10:42

Https request with mutual authentication passes with curl but fails with java

1 Answers1