I have deployed a WAR in Tomcat (TomEE 7.0 plume) with client authentication. It works well on Windows. Then I have installed the same application in Ubuntu Server, with same configuration: TomEE 7 plume, a copy of the same keystore file (JKS), etc. except that I am using OpenJDK 8 in Linux.
When I access the Ubuntu application, the HTTPS works and the page is rendered, but the problem is that even though I have some correct client certificates in my Chrome browser, the client is never authenticated from Ubuntu - as it is from Windows. I have checked some answers (1, 2, etc) and activated the debug of the ssh handshake, but the information is not very clear about what is happening. The strange thing is that it works on Windows.
I have tried to create a new JKS from Ubuntu, remove the ciphers attribute in server.xml, access the application via DNS instead of via IP, force the clientAuth with true, capture traffic with Wireshark, disable the firewall, etc and nothing solves my issue.
Any idea of what it is happening or how I can continue investigating?
server.xml
<?xml version='1.0' encoding='utf-8'?>
<Server port="8005" shutdown="SHUTDOWN">
<Listener className="org.apache.tomee.catalina.ServerListener" />
<Listener className="org.apache.catalina.startup.VersionLoggerListener" />
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
<GlobalNamingResources>
<Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<Service name="Catalina">
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<Connector port="8443"
protocol="org.apache.coyote.http11.Http11NioProtocol"
SSLEnabled="true"
maxThreads="150"
scheme="https"
secure="true"
clientAuth="want"
keystoreFile="...."
keystorePass="..."
keystoreType="JKS"
truststoreFile="..."
truststorePass="..."
truststoreType="JKS"
keyAlias="..."
sslProtocol="TLSv1.2"
ciphers="TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384,
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA"
/>
<Engine name="Catalina" defaultHost="localhost">
<Realm className="org.apache.catalina.realm.LockOutRealm">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase" />
</Realm>
<Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true">
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log" suffix=".txt" pattern="%h %l %u %t "%r" %s %b" />
</Host>
</Engine>
</Service>
</Server>
catalina.out
Unsupported extension status_request, data: 01:00:00:00:00
Unsupported extension type_18, data:
Unsupported extension type_16, data: 00:0c:02:68:32:08:68:74:74:70:2f:31:2e:31
Unsupported extension type_30032, data:
Extension ec_point_formats, formats: [uncompressed]
Extension elliptic_curves, curve names: {unknown curve 47802, unknown curve 29, secp256r1, secp384r1}
Unsupported extension type_6682, data: 00
***
%% Resuming [Session-3, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256]
*** ServerHello, TLSv1.2
RandomCookie: GMT: 1489940010 bytes = { 198, 114, 3, 85, 160, 14, 76, 230, 53, 37, 73, 132, 206, 247, 46, 1, 213, 78, 208, 199, 114, 43, 160, 223, 203, 50, 59, 176 }
Session ID: {89, 207, 174, 40, 63, 149, 210, 218, 166, 112, 193, 38, 230, 210, 151, 115, 144, 123, 12, 40, 255, 99, 212, 50, 77, 71, 11, 130, 28, 102, 183, 196}
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
Cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
CONNECTION KEYGEN:
Client Nonce:
0000: 71 BB D2 14 B7 27 4A E0 8C E5 93 59 DF 77 AB F4 q....'J....Y.w..
0010: 25 AF F3 10 CF 9F 57 85 30 A9 47 48 7C F2 D6 0E %.....W.0.GH....
Server Nonce:
0000: 59 CF AE 2A C6 72 03 55 A0 0E 4C E6 35 25 49 84 Y..*.r.U..L.5%I.
0010: CE F7 2E 01 D5 4E D0 C7 72 2B A0 DF CB 32 3B B0 .....N..r+...2;.
Master Secret:
0000: 0C 5D 4F B6 F3 6B 86 16 4C 42 7A D2 9A 99 30 60 .]O..k..LBz...0`
0010: D8 A6 A9 F1 AD 6D 36 25 C3 ED F2 2D 81 AC F0 EF .....m6%...-....
0020: 65 B5 DC EC D8 F4 70 F9 27 FE 82 43 74 E9 80 DB e.....p.'..Ct...
... no MAC keys used for this cipher
Client write key:
0000: 5B 42 51 E7 E9 CC 16 45 1B 72 74 B1 DD A6 1D 4C [BQ....E.rt....L
Server write key:
0000: F2 6C EC 51 51 C7 8F 91 DE 4A 8F 9D B1 E2 E8 70 .l.QQ....J.....p
Client write IV:
0000: 9A DE B5 3D ...=
Server write IV:
0000: 46 4E C2 71 FN.q
https-jsse-nio-8443-exec-9, WRITE: TLSv1.2 Handshake, length = 81
https-jsse-nio-8443-exec-9, WRITE: TLSv1.2 Change Cipher Spec, length = 1
*** Finished
verify_data: { 161, 183, 254, 210, 215, 177, 251, 97, 250, 132, 244, 2 }
***
*** Finished
verify_data: { 110, 171, 171, 236, 64, 1, 31, 53, 61, 163, 76, 102 }
***
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
https-jsse-nio-8443-exec-5, READ: TLSv1 Handshake, length = 206
*** ClientHello, TLSv1.2
RandomCookie: GMT: -922731054 bytes = { 232, 173, 192, 225, 198, 168, 3, 142, 225, 47, 141, 159, 160, 10, 207, 57, 139, 17, 247, 190, 186, 222, 214, 236, 131, 133, 237, 134 }
Session ID: {89, 207, 174, 40, 63, 149, 210, 218, 166, 112, 193, 38, 230, 210, 151, 115, 144, 123, 12, 40, 255, 99, 212, 50, 77, 71, 11, 130, 28, 102, 183, 196}
Cipher Suites: [Unknown 0xca:0xca, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, Unknown 0xcc:0xa9, Unknown 0xcc:0xa8, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA]
Compression Methods: { 0 }
Unsupported extension type_14906, data:
Extension renegotiation_info, renegotiated_connection: <empty>
Unsupported extension type_23, data:
Unsupported extension type_35, data:
Extension signature_algorithms, signature_algorithms: SHA256withECDSA, Unknown (hash:0x8, signature:0x4), SHA256withRSA, SHA384withECDSA, Unknown (hash:0x8, signature:0x5), SHA384withRSA, Unknown (hash:0x8, signature:0x6), SHA512withRSA, SHA1withRSA
Unsupported extension status_request, data: 01:00:00:00:00
Unsupported extension type_18, data:
Unsupported extension type_16, data: 00:0c:02:68:32:08:68:74:74:70:2f:31:2e:31
Unsupported extension type_30032, data:
Extension ec_point_formats, formats: [uncompressed]
Extension elliptic_curves, curve names: {unknown curve 2570, unknown curve 29, secp256r1, secp384r1}
Unsupported extension type_56026, data: 00
***
%% Resuming [Session-3, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256]
*** ServerHello, TLSv1.2
RandomCookie: GMT: 1489940010 bytes = { 226, 163, 151, 79, 83, 106, 222, 200, 87, 130, 48, 32, 110, 60, 118, 24, 85, 92, 147, 12, 221, 79, 6, 21, 9, 31, 37, 171 }
Session ID: {89, 207, 174, 40, 63, 149, 210, 218, 166, 112, 193, 38, 230, 210, 151, 115, 144, 123, 12, 40, 255, 99, 212, 50, 77, 71, 11, 130, 28, 102, 183, 196}
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
Cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
CONNECTION KEYGEN:
Client Nonce:
0000: C9 00 3E D2 E8 AD C0 E1 C6 A8 03 8E E1 2F 8D 9F ..>........../..
0010: A0 0A CF 39 8B 11 F7 BE BA DE D6 EC 83 85 ED 86 https-jsse-nio-8443-exec-2, WRITE: TLSv1.2 Application Data, length = 16384
https-jsse-nio-8443-exec-9, WRITE: TLSv1.2 Handshake, length = 40
verify_data: { 132, 34, 95, 186, 15, 151, 105, 43, 141, 255, 99, 200 }
***
*** Finished
verify_data: { 3, 16, 246, 54, 2, 1, 152, 125, 207, 244, 145, 208 }
***
...9............
Server Nonce:
0000: 59 CF AE 2A E2 A3 97 4F 53 6A DE C8 57 82 30 20 Y..*...OSj..W.0
0010: 6E 3C 76 18 55 5C 93 0C DD 4F 06 15 09 1F 25 AB n<v.U\...O....%.
Master Secret:
0000: 0C 5D 4F B6 F3 6B 86 16 4C 42 7A D2 9A 99 30 60 .]O..k..LBz...0`
0010: D8 A6 A9 F1 AD 6D 36 25 C3 ED F2 2D 81 AC F0 EF .....m6%...-....
0020: 65 B5 DC EC D8 F4 70 F9 27 FE 82 43 74 E9 80 DB e.....p.'..Ct...
... no MAC keys used for this cipher
Client write key:
0000: 45 8C B0 36 8B FA D2 4B 83 BA 90 D8 75 3F E1 B1 E..6...K....u?..
Server write key:
0000: BC 91 84 F4 9B DB 5C EC F8 05 AE A6 A3 48 BA 7D ......\......H..
Client write IV:
0000: 9C E7 B8 DA ....
Server write IV:
0000: 5C 36 F2 DE \6..
https-jsse-nio-8443-exec-5, WRITE: TLSv1.2 Handshake, length = 81
https-jsse-nio-8443-exec-5, WRITE: TLSv1.2 Change Cipher Spec, length = 1
*** Finished
verify_data: { 181, 50, 10, 60, 137, 228, 78, 140, 68, 204, 185, 248 }
***
https-jsse-nio-8443-exec-5, WRITE: TLSv1.2 Handshake, length = 40
https-jsse-nio-8443-exec-2, WRITE: TLSv1.2 Application Data, length = 532
https-jsse-nio-8443-exec-9, READ: TLSv1.2 Change Cipher Spec, length = 1
https-jsse-nio-8443-exec-9, READ: TLSv1.2 Handshake, length = 40
*** Finished
verify_data: { 146, 99, 0, 148, 63, 211, 82, 244, 225, 111, 29, 146 }
***
https-jsse-nio-8443-exec-2, WRITE: TLSv1.2 Application Data, length = 5703
30-Sep-2017 07:46:02.193 FINE [https-jsse-nio-8443-exec-2] sun.reflect.NativeMethodAccessorImpl.invoke Error trying to obtain a certificate from the client
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:431)
at org.apache.tomcat.util.net.jsse.JSSESupport.getPeerCertificateChain(JSSESupport.java:97)
at org.apache.coyote.AbstractProcessor.populateSslRequestAttributes(AbstractProcessor.java:597)
at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:358)
at org.apache.coyote.Request.action(Request.java:393)
at org.apache.catalina.connector.Request.getAttribute(Request.java:900)
at org.apache.catalina.connector.Request.getAttributeNames(Request.java:982)
at com.sun.faces.application.WebappLifecycleListener.requestDestroyed(WebappLifecycleListener.java:114)
at com.sun.faces.config.ConfigureListener.requestDestroyed(ConfigureListener.java:383)
at org.apache.catalina.core.StandardContext.fireRequestDestroyEvent(StandardContext.java:5946)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:182)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at org.apache.tomee.catalina.OpenEJBSecurityListener$RequestCapturer.invoke(OpenEJBSecurityListener.java:97)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:624)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:798)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1434)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
https-jsse-nio-8443-exec-7, called closeOutbound()
https-jsse-nio-8443-exec-7, closeOutboundInternal()
https-jsse-nio-8443-exec-7, SEND TLSv1.2 ALERT: warning, description = close_notify
https-jsse-nio-8443-exec-7, WRITE: TLSv1.2 Alert, length = 26
https-jsse-nio-8443-exec-6, called closeOutbound()
https-jsse-nio-8443-exec-6, closeOutboundInternal()
https-jsse-nio-8443-exec-6, SEND TLSv1.2 ALERT: warning, description = close_notify
https-jsse-nio-8443-exec-6, WRITE: TLSv1.2 Alert, length = 26
Keystore listing
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 4 entries
Alias name: .....
Creation date: Sep 20, 2017
Entry type: trustedCertEntry
Owner: CN=...
Issuer: CN=..., OU=..., O=...
Serial number: 5d305b71
Valid from: Wed Sep 20 08:26:20 PDT 2017 until: Thu Mar 19 08:26:20 PDT 2037
Certificate fingerprints:
MD5: AB:C2:0C:21:C5:5C:F2:D6:69:30:4F:76:7B:AD:74:D4
SHA1: 2D:B2:83:86:3C:E7:AE:1B:6A:2B:1D:A2:F5:D2:BF:CE:5A:4D:A9:AF
SHA256: 7F:FF:C2:AD:A6:AC:32:58:14:04:EA:E7:6B:F1:01:C8:3E:64:21:85:D5:54:F6:99:1A:07:AE:4A:9B:A2:26:23
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
clientAuth
]
#2: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
URIName: ...
]
*******************************************
*******************************************
Alias name: ....
Creation date: Oct 2, 2017
Entry type: trustedCertEntry
Owner: CN=...
Issuer: CN=..., OU=..., O=....
Serial number: 28f991cc
Valid from: Mon Oct 02 02:36:37 PDT 2017 until: Tue Mar 31 02:36:37 PDT 2037
Certificate fingerprints:
MD5: D0:83:0C:77:64:DE:04:A2:87:3A:99:6D:28:87:83:FD
SHA1: 63:C7:84:6F:97:97:DA:98:38:38:34:97:16:A8:38:78:28:95:82:C7
SHA256: FA:A5:36:D1:59:A2:E7:C0:55:6F:57:F9:39:48:4D:9E:AE:5C:39:66:CB:CF:B0:83:4B:1F:72:55:E2:6E:81:16
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
clientAuth
]
#2: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
URIName: .....
]
*******************************************
*******************************************
Alias name: ...
Creation date: Sep 20, 2017
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=..., OU=..., O=...
Issuer: CN=..., OU=..., O=...
Serial number: 71ed7b34
Valid from: Wed Sep 20 08:26:19 PDT 2017 until: Thu Mar 19 08:26:19 PDT 2037
Certificate fingerprints:
MD5: 90:DD:A1:A9:E0:16:46:97:43:88:9C:C4:06:FC:46:65
SHA1: 31:9D:ED:05:EC:53:89:19:E8:46:93:9D:69:E0:49:44:9D:3F:40:7E
SHA256: D0:78:96:D6:D6:BF:7F:DC:2D:74:7A:B7:AA:A4:E1:17:9A:56:37:68:A4:C9:C5:2F:BE:87:6C:7C:B8:D9:E3:23
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:true
PathLen:0
]
#2: ObjectId: 2.5.29.30 Criticality=true
NameConstraints: [
Permitted: GeneralSubtrees:
[
GeneralSubtree: [
GeneralName: DNSName: ....
Minimum: 0 Maximum: 0 ]
]
]
#3: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: localhost
DNSName: ....
IPAddress: 127.0.0.1
]
....
EDIT
After an awful amount of testing, I have copied directly my TomEE Windows version (apache-tomee-7.0.0-plume) as it is into the Linux server. By doing this, I have to do more tests but it seems to work as expected. So I checked that Linux was using "apache-tomee-plume-7.0.3" and I wonder if there might be a bug in that version (?)
