We are trying to get TLS1.2 working in FIPS mode in the following environment. When attempting to write to an SSLSocket created with a SSLContext for protocol “TLSv1.2” it fails with the error java.security.NoSuchAlgorithmException: no such algorithm: SunTls12RsaPremasterSecret for provider SunPKCS11-NSS.
The environment:
- Java: OpenJDK 1.8.0_25-b17 (client and server)
- OS: CentOS release 6.5 (Final)
- NSS: nss-3.16.2.3-3.el6_6.x86_64
- FIPS mode is enabled on keystore using modutil.
I believe the answer is NO, TLS1.2 does not work with NSS 3.16+ in FIPS mode. I need to determine decisively:
- whether this is expected to work and I should therefore continue digging to find out what we are doing wrong (in which case pointers and ideas to make this work are requested) or
- it is not supported and I should therefore expect it to be supported in which future NSS version
Here is the evidence collected so far:
- The error that occurs when trying to use TLS1.2 is java.security.NoSuchAlgorithmException: no such algorithm: SunTls12RsaPremasterSecret for provider SunPKCS11-NSS. An identical error message is disussed in this post, with indications that it means TLS 1.2 is not supported.
- FIPS validation of NSS was last performed in 2010 on NSS version 3.12.14. TLS1.2 support is not mentioned in any NSS release notes until 3.15.1. A related question is posted here, but there is no specific mention of TLSv1.2.
- CentOS 6.5 release notes indicate that FIPS mode and TLS1.2 are supported, but there is no mention of whether the two are supported at the same time.
- Our tests using TLS1.2 SSLContext (client and server) work when the system is NOT in FIPS mode. Running the exact same test after running modutil –fips true results in the described error and console output provided below.
The relevant javax.net.debug output follows:
*** ClientHello, TLSv1.2
RandomCookie: GMT: 1409235389 bytes = { 122, 104, 164, 187, 130, 152, 6, 95, 250, 230, 146, 99, 164, 228, 116, 203, 188, 51, 48, 140, 196, 35, 87, 33, 228, 67, 15, 120 }
Session ID: {}
Cipher Suites: [TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods: { 0 }
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
***
TRIMMED
%% Negotiating: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA256]
*** ServerHello, TLSv1.2
RandomCookie: GMT: 1409235389 bytes = { 217, 13, 11, 142, 204, 139, 77, 178, 239, 246, 177, 116, 225, 208, 217, 77, 128, 106, 206, 72, 40, 229, 46, 232, 54, 172, 74, 41 }
Session ID: {84, 255, 58, 189, 125, 207, 159, 166, 144, 6, 19, 157, 173, 113, 80, 88, 204, 69, 101, 231, 227, 242, 144, 59, 174, 147, 158, 65, 14, 78, 182, 52}
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256
Compression Method: 0
Extension renegotiation_info, renegotiated_connection:
***
Cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256
TRIMMED
*** ServerHelloDone
[read] MD5 and SHA1 hashes: len = 4
0000: 0E 00 00 00 ....
Thread-0, handling exception: javax.net.ssl.SSLKeyException: RSA premaster secret error
%% Invalidated: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA256]
12:41:01.684 [Thread-0] ERROR c.p.p.c.i.SslContextTestContainer - unexpected Exception
javax.net.ssl.SSLKeyException: RSA premaster secret error
at sun.security.ssl.RSAClientKeyExchange.(RSAClientKeyExchange.java:86) ~[na:1.8.0_25]
at sun.security.ssl.ClientHandshaker.serverHelloDone(ClientHandshaker.java:880) ~[na:1.8.0_25]
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:344) ~[na:1.8.0_25]
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:936) ~[na:1.8.0_25]
at sun.security.ssl.Handshaker.process_record(Handshaker.java:871) ~[na:1.8.0_25]
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1043) ~[na:1.8.0_25]
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343) ~[na:1.8.0_25]
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:728) ~[na:1.8.0_25]
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) ~[na:1.8.0_25]
at java.io.OutputStream.write(OutputStream.java:75) ~[na:1.8.0_25]
at com.polycom.pillars.certificate.internal.SslContextTestContainer$ClientThread.doit(SslContextTestContainer.java:193) ~[bin/:na]
at com.polycom.pillars.certificate.internal.SslContextTestContainer$SslConnectionThread.run(SslContextTestContainer.java:127) ~[bin/:na]
Caused by: java.security.NoSuchAlgorithmException: no such algorithm: SunTls12RsaPremasterSecret for provider SunPKCS11-NSS
at sun.security.jca.GetInstance.getService(GetInstance.java:101) ~[na:1.8.0_25]
at javax.crypto.JceSecurity.getInstance(JceSecurity.java:109) ~[na:1.8.0_25]
at javax.crypto.KeyGenerator.getInstance(KeyGenerator.java:287) ~[na:1.8.0_25]
at sun.security.ssl.JsseJce.getKeyGenerator(JsseJce.java:274) ~[na:1.8.0_25]
at sun.security.ssl.RSAClientKeyExchange.(RSAClientKeyExchange.java:77) ~[na:1.8.0_25]
