2

My client(java 1.8) is trying to set up a TLS connection with a remote server using TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 as cipher suite. But throws an java.lang.IllegalStateException

HashAlgorithm.sha256(4) is not being tracked

The used crypto material used shall be based on a brainpool curve. The client uses BouncyCastleJsseProvider crypto provider. Following system properties have been set

System.setProperty("jdk.tls.ephemeralDHKeySize", "2048");
System.setProperty("jdk.tls.server.defaultDHEParameters",
            "{ " + "FFFFFFFF FFFFFFFF C90FDAA2 2168C234 C4C6628B 80DC1CD1 "
                    + "29024E08 8A67CC74 020BBEA6 3B139B22 514A0879 8E3404DD "
                    + "EF9519B3 CD3A431B 302B0A6D F25F1437 4FE1356D 6D51C245 "
                    + "E485B576 625E7EC6 F44C42E9 A637ED6B 0BFF5CB6 F406B7ED "
                    + "EE386BFB 5A899FA5 AE9F2411 7C4B1FE6 49286651 ECE45B3D "
                    + "C2007CB8 A163BF05 98DA4836 1C55D39A 69163FA8 FD24CF5F "
                    + "83655D23 DCA3AD96 1C62F356 208552BB 9ED52907 7096966D "
                    + "670C354E 4ABC9804 F1746C08 CA18217C 32905E46 2E36CE3B "
                    + "E39E772C 180E8603 9B2783A2 EC07A28F B5C55DF0 6F4C52C9 "
                    + "DE2BCBF6 95581718 3995497C EA956AE5 15D22618 98FA0510 "
                    + "15728E5A 8AACAA68 FFFFFFFF FFFFFFFF, 2}");
System.setProperty("jdk.tls.namedGroups", "brainpoolp256r1,brainpoolP384r1");

For debugging purposes the handshake has been sniffed with WireShark:

WireShark.

It can be seen in the screenshot that the client is requesting the correct cipher suite. Client Hello - Signature hash algorithms: Client Hello - Signature hash algorithms

The server accepts the suite and the handshake is handled up until the client generates server certificates verification message, which leads into the previously mentioned java.lang.IllegalStateException. Since the exception message reads 'HashAlgorithm.sha256(4) is not being tracked'. I am assuming that, for some reason, the client does not use sha384 for the generation of 'certificate Verify' but sha256.

I am assuming that I might have missed some client configuration step, to make sha384 available. Sadly I've not found a hint which could provide me with some helpful information. Can someone reach me a helping hand in that question?

P.S. If TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 is chosen instead of TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 the handshake operation is successful, but this suite does not make use of sha384 for calculating the contents of the certificate verification message.

P.S.S Since it seems that the client does not provide Signature Hash algorithms for (sha256(4), ecdsa(3)) and (sha384(5), ecdsa(3)) as stated in https://www.rfc-editor.org/rfc/rfc5246#section-7.4.1.4.1, is there some option to programmatically provide support of these algorithms?

P.S.S.S. Screenshot of Server Certificate Request WireShark protocol added. Server - Certificate Request

Community
  • 1
  • 1
Christoph Bröter
  • 21
  • 4
  • Let me guess: You are using OpenJDK on Linux? ECDSA is known to be a problem with OpenJDK on Linux. – Robert Jan 06 '20 at 09:53
  • Currently I am using OpenJDK but on a Windows10 machine. Thx for the reply. – Christoph Bröter Jan 06 '20 at 10:03
  • What's the Java version you are using? `java -version` output please. – Karol Dowbecki Jan 06 '20 at 10:13
  • openjdk version "1.8.0_222" OpenJDK Runtime Environment (build 1.8.0_222-b10) Eclipse OpenJ9 VM (build openj9-0.15.1, JRE 1.8.0 Windows 10 amd64-64-Bit Compressed References 20190717_421 (JIT enabled, AOT enabled) OpenJ9 - 0f66c6431 OMR - ec782f26 JCL - f147086df1e based on jdk8u222-b10) – Christoph Bröter Jan 06 '20 at 10:24
  • Looks like there is no ECDSA in plain OpenJDK 1.8. Then Windows was just no problem because everybody was using OracleJDK. If you can, use OpenJDK11 instead, there ECDSA is included. Or include BouncyCastle security provider in your server. According to some older Stackoverflow answers this should also make OpenJDK 1.8 capable of using ECDSA. – Robert Jan 06 '20 at 12:09
  • Thank you for your reply. I am already making use of BouncyCastle as a provider by using BouncyCastleJsseProvider to enable ECDSA support on OpenJDK 1.8. Sadly I am not in control of the Server, thus i can not fiddle around with the connection. – Christoph Bröter Jan 06 '20 at 13:05
  • (0) using BouncyCastleJsse (bctls) by itself doesn't provide EC support. It implements the EC _suites_, but so does SunJSSE in j6 (yes 6) up; either version of Jsse requires an underlying provider for the EC _primitives_, which Sun did not in j6 (only j7 up) and openjdk might not (I'm not so sure of that). You might need the Bouncy 'base' provider (bcprov) as well as bctls. But you certainly have some provider because otherwise the ciphersuite would not be offered. (1) setting DHE parameters, or explicit size, in _client_ is and has always been useless. .... – dave_thompson_085 Jan 06 '20 at 15:25
  • (2) your client is indeed offering sigalgs 0x0403 and 0x0503, but wireshark is decoding them in TLS1.3 format (as a single 2-byte value instead of separate bytes) and with the TLS1.3 more restrictive meanings, both wrong for your request which is TLS1.2. But this is irrelevant; sigalgs in clienthello only affects signature by _server_ not _client_; _client_ signature is controlled by the sigalgs field in server's CertificateRequest message, which you didn't show. (3) what version of Bouncy? I could try to recreate. – dave_thompson_085 Jan 06 '20 at 15:30
  • Thanks @dave_thompson_085 that clarified a lot. I've added the Servers Certificate Request to the post. (0)bcprov already has been used implicitly I did not consider it to be important information, thx. (1) Thanks for the headsup, will remove it. (3) Am using bcprov-jdk15to18-164, and bctls-jdk15to18-164 – Christoph Bröter Jan 07 '20 at 10:38

0 Answers0