Apache SSL Kafka on Windows | PKIX path building failed

Question

I am trying to access kafka topic on Windows locally via my Eclipse java code. The kafka topic is SSL secured. I have a keystore and certificate (.cer) files as well, which i got it from the other downstream Team.

SSl params being used are below
prop.put("security.protocol", "SSL"); 
prop.put("ssl.keystore.location",${unix or Windows path}); 
prop.put("ssl.keystore.password", password);

I am able to access the kafka topic when i build my jar and deploy it to the unix box and run it via java -cp, etc cmd. i input the keystore location as say for example -

/tmp/keystore.jks

The problem here is, i want to access the same SSL kafka topic locally on my Windows as well, so i am trying to input below keystore location as for example (i have the keystore available locally on below Windows Path) -

C:\\userID\\Desktop\\keystore.jks

But i get the error of

Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The kafka debug logs show that its picking the correct keystore file but still failing

ssl.keystore.location = C:\userID\Desktop\keystore.jks
ssl.keystore.password = [hidden]
ssl.keystore.type = JKS

The issue here is, i even tried to add the cer file to my Java local via keytool import command, but i do not have Admin access to change the Program Files Java cacerts file. As a result, i get access denied error.

I even tried below in my main class method but it did not work. Even i tried it to pass it in the -D params as arguments, but failed.

System.setProperty("javax.net.ssl.keyStore","C:\\userID\\Desktop\\keystore.jks");
System.setProperty("javax.net.ssl.keyStorePassword",password);

Is there a way to get around it as ultimately i want to build a Java executable Windows App which can connect to SSL kafka topics and distribute to my entire team the executable Java app.

Updated debug logs from 
-Djavax.net.debug=ssl

javax.net.ssl|FINE|01|main|2022-07-31 11:10:33.097 EDT|SSLCipher.java:438|jdk.tls.keyLimits:  entry = AES/GCM/NoPadding KeyUpdate 2^37. AES/GCM/NOPADDING:KEYUPDATE = 137438953472
javax.net.ssl|SEVERE|01|main|2022-07-31 11:10:33.945 EDT|TransportContext.java:361|Fatal (CERTIFICATE_UNKNOWN): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (
"throwable" : {
  sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
    at sun.security.validator.Validator.validate(Validator.java:271)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:312)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:275)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:140)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:630)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:471)
    at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:367)
    at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:376)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:479)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:990)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:977)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:924)
    at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:336)
    at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:417)
    at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:270)
    at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:69)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:360)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:313)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:349)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:226)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:188)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:210)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:196)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:281)
    at org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1030)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:996)
  Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
    ... 30 more}

)
javax.net.ssl|WARNING|01|main|2022-07-31 11:10:33.946 EDT|SSLEngineOutputRecord.java:173|outbound has closed, ignore outbound application data

Any Help is appreciated !! Thanks

_able to access the kafka topic when i build my jar and deploy it to the unix box_... Then are you able to do the same in WSL2 from windows? Alternatively, Java can be installed in any folder, doesn't have to be in "program files" if you really need to modify the cacerts — OneCricketeer, Jul 31 '22 at 14:05
@OneCricketeer Sorry, i tried to access WSL, but thats not accessible to me as my VM has limited access. I am really not looking to get into cacerts file etc. Actually, i am just looking for somehow my consumer config properties to pick up the custom keystore file path in Windows which i am providing it, but thats not happening right now. What i am thinking is kafka does not allow to use Windows path, only unix paths in the prop.put("ssl.keystore.location",${unix or Windows path}); , thats why its not picking up my keystore, i think. No ? — username89, Jul 31 '22 at 14:12
I typically don't develop on windows. The Java Properties object doesn't care about what OS you're on, but Windows paths do need to use double backslash instead — OneCricketeer, Jul 31 '22 at 14:17
yes right. Oh sorry, i will edit it in my question, as i am using double slashes everywhere in my Java code to pass the Windows path, apologies, let me edit it quick. The thing is, i am able to easily access the Non-SSL kafka topics via Windows executable java code, but SSL is a blocker for me in Windows. I have a JSwings java executable Windows App right now, which my entire team is using like a Windows double click App, i am already doing Non-SSL kafka topics consumer operations via it, but SSL not able to figure out due to the keystore issues :/ — username89, Jul 31 '22 at 14:22
Try including debug output https://stackoverflow.com/a/9210661/2308683 — OneCricketeer, Jul 31 '22 at 14:25
@OneCricketeer Thanks once again !! Updated the debug error logs. — username89, Jul 31 '22 at 14:32
Doesn't really give much extra details. I'm not very familiar with Java ssl options, but everytime I've needed to use a keystore with Kafka, I also needed a truststore. Why do you not need it here? — OneCricketeer, Jul 31 '22 at 15:30
@OneCricketeer , i understand but due to some reason, only having keystore is working for us. I had ran the same code with unix file path with only keystore location/password specified and nothing for truststore, i saw the INFO kafka logs also printed with trustore location/password = null and it worked good on the unix box. Same code on Windows doesnt work with windows filepath :/ I had even checked with my another Team member, he said he is also using just keystore and its working fine for him on the unix server. — username89, Aug 02 '22 at 21:00
@OneCricketeer , damn, u were right !! When i specified the truststore option in the Java setSystem property rather than the kafka SSL properties, it worked !! Thank you so much !! — username89, Aug 02 '22 at 22:39

score 2 · Answer 1 · edited Aug 05 '22 at 03:15

Apparently, setting the truststore properties via below commands worked for me on Windows! The same commands did not work when I was setting them via the Kafka properties.

Just FYI, get the truststore and keystore or generate them for the new servers to access. I got both the truststore and keystore from my downstream Team.

System.setProperty("javax.net.ssl.trustStore", "C:\\Users\\userID\\cacerts.jks");
System.setProperty("javax.net.ssl.trustStorePassword", "xxxxxxx");

For the Kafka SSL properties, I have specified the lines below:

props.put("security.protocol", "SSL");
props.put("ssl.keystore.location","C:\\Users\\UserID\\keystore.jks"); 
props.put("ssl.keystore.password", "xxxxxxxx");

Hope it helps!

Apache SSL Kafka on Windows | PKIX path building failed

1 Answers1