5

I have two CentOS 6.7 machines (server A and B). Same package versions are installed on each system.

Last week I enabled the openSSH FIPS 140-2 module on Server A, and that system works flawlessly (including tsql queries to a SQL Server instance).

Today I went through the same steps on Server B (in the link above). After rebooting, fips showed enabled and tested OK - but tsql (to the same SQL Server instance) stopped working with the following error: 

[serverB ~]# tsql -S egServer80 -U myusername
Password: 
locale is "en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
Error 20002 (severity 9):
    Adaptive Server connection failed
There was a problem connecting to the server

I check the log files and find this: 

tsql: Libgcrypt warning: MD5 used - FIPS mode inactivated

Enabling debug in freetds yielded this additional error: 

14:56:46.617196 3577 (net.c:1366):'''handshake failed: GnuTLS internal error.

Backing out the FIPS module (removing fips=1 from grub.conf) and rebooting set things back and I was again able to tsql into my SQL Server instance.

Both CentOS machines use the same libgcrypt version (1.4.5).

Why (or how) is enabling FIPS in grub causing libgcrypt to fail on this one machine? Clearly something configuration is off between the machines, but I cannot spot it, and have exhausted resources in finding where to look next.

NOTE #1:

I can reproduce the tsql problem on Server B without enabling FIPS 140-2 in grub, by creating an empty file /etc/gcrypt/fips_enabled. Removing this file sets the system back to normal, and tsql works again.

NOTE #2:

Adding file /etc/gcrypt/fips_enabled on Server A does not affect tsql function at all.

Additional Information

libgcrypt version 1.4.5
freetds version 0.91
openssl version 1.0.1e
CentOS version 6.7
Stuart J Cuthbertson
  • 390
  • 5
  • 13
a coder
  • 7,530
  • 20
  • 84
  • 131
  • Different certificates on the SQL servers? Different configuration on cipher suites? – Etan Reisner Aug 18 '15 at 18:02
  • I'm connecting to the same SQL server instance from both systems. One connects. The other shows the libgcrypt warning and fails. I'll double check cipher suites, but I believe they are identical. – a coder Aug 18 '15 at 19:20
  • Get packet captures from the two connection attempts and compare where the connections differ? Presumably somewhere in the handshake/etc. – Etan Reisner Aug 18 '15 at 19:21
  • Right. Should libgcrypt not be using MD5 when FIPS is enabled though? – a coder Aug 18 '15 at 19:31
  • I think that's exactly the problem. Something is forcing MD5 usage so FIPS can't be enabled so the connection gets terminated. The question is why is MD5 usage being forced/enabled in the first place. (I'm assuming MD5 is not FIPS-compliant because... well because it is MD5 and because the error leads me to believe that. I don't know for sure that it isn't.) – Etan Reisner Aug 18 '15 at 19:45
  • MD5 is *not* FIPS compliant. :) I compared cipher lists using `openssl ciphers -v 'HIGH:MEDIUM:!ADH'` and see a dozen or so differences. I'll follow up shortly. – a coder Aug 18 '15 at 19:49
  • The question isn't ciphers available it is ciphers configured for actual use on the server and on the client as well as what ends up being negotiated. Also potentially what the keys were created with. – Etan Reisner Aug 18 '15 at 20:35
  • Can you run an strace tsql for the one that is failing? At what point, syscall is it failing? – Rico Aug 19 '15 at 06:40
  • I ran it, here's the last bit right before the error log is written and exit: http://justpaste.it/n4xs – a coder Aug 19 '15 at 16:06
  • _MD5_ hash is not allowed in _FIPS_ mode (_SHA1_ will most likely have the same faith). Here's what happens in _OpenSSL_'s case: `OPENSSL_FIPS=1 ${OPENSSL_INSTALL_DIR}/bin/openssl md5 ${OPENSSL_INSTALL_DIR}/bin/openssl` yields: `40242339776192:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:fips_md.c:180`. – CristiFati Sep 14 '16 at 15:27

0 Answers0