0

Java:1.8 Kafka runs in AWS as managed service MSK. cluster is enabled with client auth required. Kafka client setup via Spring boot as consumer/producer. Few ref links related: https://kafka.apache.org/documentation/#adminclientconfigs Keys were generated by aws certificate manager.

My client configs:

ssl.keystore.type:PEM
ssl.keystore.certificate.chain:"BEGIN CERTIFICATE....\r\n...END CERTIFICATE"
ssl.keystore.key:"BEING ENCRYPTED PRIVATE KEY...\r\n...END ENCRYPTED PRIVATE KEY"
ssl.key.password: "pass"

Issue: Apache Kafka Java/Springboot Client with PEM certs fails to load private key. It works with JKS created out of these PEM files via

 ssl.keystore.location=path, ssl.truststore.location=path,
ssl.keystore.type=JKS, ssl.truststore.type=JKS, ssl.keystore.password,ssl.truststore.password.

Trying to make it connect with PEM cert chain/key/keypass but my code fails to read private key. In need of this change so that PEM can be safely provided as env vars via valut/credhub etc.

Any insights would be really helpful. any inputs needed let me know. thanks a lot!

Log snippet:

2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: Invalid PEM keystore configs
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] Caused by: java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 48)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at sun.security.util.ObjectIdentifier.<init>(ObjectIdentifier.java:285)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at sun.security.util.DerInputStream.getOID(DerInputStream.java:320)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:137)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at sun.security.x509.AlgorithmId.<init>(AlgorithmId.java:119)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:393)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:95)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.privateKey(DefaultSslEngineFactory.java:512)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.createKeyStoreFromPem(DefaultSslEngineFactory.java:462)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory$PemStore.<init>(DefaultSslEngineFactory.java:435)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.createKeystore(DefaultSslEngineFactory.java:284)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at org.apache.kafka.common.security.ssl.DefaultSslEngineFactory.configure(DefaultSslEngineFactory.java:161)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at org.apache.kafka.common.security.ssl.SslFactory.instantiateSslEngineFactory(SslFactory.java:136)
2021-11-01T12:03:27.529-04:00 [APP/PROC/WEB/0] [OUT] at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:93)
Muthuponmozhi Somasundaram
  • 1
  • 2
  • What version of Java? Your stacktrace looks like 8, and versions below 11.0.1 had a bug with exactly this symptom parsing parameters for PBES2 encryption which your keyfile apparently uses; see dupe https://stackoverflow.com/questions/51883324/why-can-encryptedprivatekeyinfo-not-read-my-pkcs8-encrypted-private-key-in-java and https://bugs.openjdk.java.net/browse/JDK-8076999 which initially was fixed only in 11.0.1 up but https://bugs.openjdk.java.net/browse/JDK-8202837 (now) claims a backport to 8u301 (and 7u311). – dave_thompson_085 Nov 01 '21 at 18:59
  • Please provide enough code so others can better understand or reproduce the problem. – Community Nov 02 '21 at 08:47
  • @dave_thompson_085 Thanks for your time. Yes its 1.8. Added few more details as well now to my Qs. I also did read the error mentioned in Java site. But not sure if that is my issue due to my less knowledge behind encryption/key/algorigthms/terms etc. Is protected key can not be read by Java default ssl engine factory class? – Muthuponmozhi Somasundaram Nov 06 '21 at 21:19
  • Which (1.)8? You are not (yet) in any 'Java default SSL engine' -- you are in kafka code calling `javax.crypto.EncryptedPrivateKeyInfo` which is part of basic Java cryptography not specific to SSL/TLS. And that code had a bug in handling _some_ PKCS8 encryption schemes, specifically those using PBES2 from PKCS5v2, and thus failed for keys encrypted using those schemes while working for other schemes. As I said this was **fixed (only) in 11.0.1 8u301 7u311 up**; I have now confirmed 8u301. – dave_thompson_085 Nov 08 '21 at 03:41
  • Thank you @Dave for providing this input and for correcting me. Let me update my team about this. – Muthuponmozhi Somasundaram Nov 09 '21 at 18:57

0 Answers0