2

I've found some related questions, but mine is a little different.

Client app running on Windows, using Java 1.7 (71) with bouncycastle connecting to a tomcat 8.5.xx server running Java 1.8 (161) on either Linux RH or Windows 10. The Linux and Windows server installations should be identical (other than being Linux and Windows). We control the software on both sides.

Client app works correctly when connecting to Windows. When connecting to Linux it fails the SSL handshake with:

java.security.InvalidAlgorithmParameterException: Prime size must be multiple of 64, and can only range from 512 to 1024 (inclusive)
  at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DHKeyPairGenerator.java:120)
  at java.security.KeyPairGenerator$Delegate.initialize(Unknown Source)

This isn't totally surprising, because Java 7 can't handle large DH keys. But when the client connects to the Windows server (same version of tomcat, same version of Java), the SSL handshake works flawlessly and the connection goes through.

Setting javax.net.debug appropriately shows that, yes, the Linux server is sending a 2048 bit DH key and Windows is sending 1024. I have no idea why.

The java.security files are the same for the Windows and Linux server (except windows has sun.security.mscapi.SunMSCAPI provider). crypto.policy=unlimited is commented out on both.

Two part question:

  1. Where else can I look to find out why the two systems behave differently? Is this an OS crypto library difference?
  2. Is there a way to resolve this problem without changing the client code (it's legacy code and we'd rather not patch)?
jww
  • 97,681
  • 90
  • 411
  • 885
user1181015
  • 21
  • 3
  • @jww: although unlimited-policy definitely has no effect on DH or any other asymmetric crypto in Suncle. OP: it's not OS; Java rarely and Tomcat never uses OS crypto. Are you setting javax.net.debug on client or server(s)? If former, have you checked whether both servers are using JSSE (the pure-Java stack) versus OpenSSL (in tc8.5 either directly as APR or via wrapper)? Possible workaround: both j7 and j8 JSSE (if used) _should_ support and _prefer_ ECDHE over DHE, and ECDHE does not have this issue; have you checked why it isn't being used? – dave_thompson_085 Aug 14 '18 at 02:01
  • Both are using JSSE, I believe. Regardless, the configuration is the same for both (same server.xml and flags). As it happens, putting unlimited strength crypto on the client side (which already had bouncy castle installed) made the issue go away, although I still don't understand the asymmetry between our Win and Linux servers. – user1181015 Aug 15 '18 at 23:20
  • _IF_ the Connector config (in server.xml) defaults the stack rather than specifying it, that default depends on whether tcnative is installed, which differs by platform. That's why I hinted you _check_ if servers use JSSE not guess. I'm surprised 256-bit policy made a difference unless either the client is using a weird ciphersuite order (which you could see from your debug log, or client code or environment e.g. `https.ciphersuites` if HttpsURLConnection) or the server is using a weird order _and_ orverriding (both of which would be in the config). – dave_thompson_085 Aug 16 '18 at 23:55

0 Answers0