0

I'm implementing a protocol that specifies TLS1.2 as transport layer and requires client-side server authentication to verify the server's hostname by comparing the hostname value of the connecting client socket to the value indicated by the server in its certificate, namely subjectAltName extension of type dNSName.

I've created a test, put this value in the server's certificate and it seemed to get ignored by the client completely, but I'd like to be sure. Do I have to code this check in an implementation of X509ExtendedTrustManager.checkServerTrusted(X509Certificate[], String, Socket) or can I enable it through some obscure property? The reference guide appears silent on this matter.

The protocol specification (the one I'm implementing) also mentions that wildcards may be used as a prefix of the value in the certificate.

A "*" wildcard character MAY be used as the leftmost name component in the certificate. For example, *.example.com would match a.example.com, foo.example.com, etc., but would not match example.com.

However, when I tried to create such an extension value with keytool, it refused to do so. What's going on?

"C:\Program Files\Java\jdk1.7.0_51\bin\keytool.exe" -genkeypair -alias server -keyalg RSA -validity 365 -ext san=dns:*.example.com -keystore mykeystore ...

keytool error: java.lang.RuntimeException: java.io.IOException: DNSName components must begin with a letter

predi
  • 5,528
  • 32
  • 60

2 Answers2

2

Java 7 provides a way to verify the host name automatically on an SSLSocket or SSLEngine, but it is not enabled automatically (this didn't exist in Java 6). The implementation can use the naming specification of LDAP or HTTPS, not the more generic RFC 6125 (at least not yet). In most cases, using this part of the HTTPS specification should be fine for other protocols, certainly better than nothing.

You can use this as follows:

SSLParameters sslParams = new SSLParameters();
sslParams.setEndpointIdentificationAlgorithm("HTTPS");
sslSocket.setSSLParameters(sslParams); // or SSLEngine

You can find references for this:

The second problem you have seems to be an issue with keytool, to generate such a certificate. This doesn't affect how such certificates presented to a Java client are verified. You can use other tools to generate a certificate with a wildcard if necessary.

EDIT:

Note that for this to work, you need to create the SSLSocket with a method that uses String host (so that it knows the host name) or the SSLEngine with SSLContext.createSSLEngine(String peerHost, int peerPort). This is how it knows which hostname to try to match in the certificate. (Using the name is also useful for SNI.)

Bruno
  • 119,590
  • 31
  • 270
  • 376
  • Hmm.. The reference guide seems to be mighty misleading on this topic. Can I react when the above logic fails, like with the `HttpsURLConnection` example? It uses an implementation of `HostnameVerifier` interface for this. – predi Sep 03 '14 at 08:38
  • The documentation isn't very clear indeed. Sure, you can catch the exception if you want, you can implement and use you own `X509ExtendedTrustManager` on top of the other settings if you wish, as shown in [the example in the JSSE Ref Guide](http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html#X509ExtendedTrustManager) (you'll just need to catch the exceptions of the original manager and act appropriately if you want). – Bruno Sep 03 '14 at 08:53
  • I've tested this and it worked. It's not that useful in my case, since the protocol I'm implementing requires a specific hostname verification algorithm (the above calls the `HostnameChecker` object mentioned by @bayou.io), and there's no point in having two implementations, but it should be useful to others. – predi Sep 03 '14 at 09:08
  • My guess is that you're trying to use the SMTP or NNTP rules. I wouldn't necessarily panic over such a detail, but you're right to be strict. You may find the `LDAP` rules actually do what you want regarding the wildcard. If you look at the [implementation](https://github.com/openjdk-mirror/jdk7u-jdk/blob/master/src/share/classes/sun/security/util/HostnameChecker.java#L246), `TYPE_LDAP` uses `matchLeftmostWildcard` (for some reason, the internal name of the constant for the `HTTPS` identification algorithm is `TYPE_TLS`, but that doesn't matter). – Bruno Sep 03 '14 at 11:42
  • Yes. The spec I'm implementing relies on NNTP ([rfc4642](http://tools.ietf.org/html/rfc4642#section-5)) rules, which are not quite the same as those implemented for `LDAP`. This implementation can match a (most specific) Common Name field instead of dNSName if SAN is not found for example. – predi Sep 03 '14 at 12:15
  • The LDAP rules will also match the most specific CN if the SAN is not found. That's what the [Java implementation does](https://github.com/openjdk-mirror/jdk7u-jdk/blob/master/src/share/classes/sun/security/util/HostnameChecker.java#L174) and that's also [in the spec](http://tools.ietf.org/html/rfc4513#section-3.1.3). – Bruno Sep 03 '14 at 12:20
  • Yes. Exactly. But my spec is supposed to match dNSNames exclusively and does not even mention CN fields. "This implementation" in my previous comment referred to the java implementation. Sorry if that was unclear. – predi Sep 03 '14 at 12:32
  • Ah indeed, RFC 4642 doesn't say anything about the CN. This being said, although I'm supportive of being strict (the "be tolerant in what you receive" principle is not actually a good idea in general), I'm not sure many NNTP clients actually do follow these rules that strictly (considering, for example that browsers don't check RFC 2818 for IP addresses properly in general). In addition, you could consider that RFC 6125 updates RFC 4642 in that respect, in which case it should be OK to accept the CN as a fallback solution. – Bruno Sep 03 '14 at 13:18
  • This is nice. However it should clarify how the expected hostname is specified. – ZhongYu Sep 03 '14 at 17:06
  • @Bruno - "browsers don't check RFC 2818 for IP addresses properly" - do you mean they don't check iPAddress in the server certificate (if the URL contains IP instead of host name)? – ZhongYu Sep 03 '14 at 17:17
  • @bayou.io You're right, I'll edit my answer. Regarding browsers and IP address, I mean that they often accept certificates that have the IP address in the CN without being in the iPAddress SAN (actually, [IE doesn't even understand iPAddress SANs](http://stackoverflow.com/a/25005558/372643)). – Bruno Sep 03 '14 at 17:28
1

The reference guide on host name verification:

http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#ciphersuitechoice

When using raw SSLSocket and SSLEngine classes, you should always check the peer's credentials before sending any data. The SSLSocket and SSLEngine classes do not automatically verify that the host name in a URL matches the host name in the peer's credentials. An application could be exploited with URL spoofing if the host name is not verified.

Therefore you have to manually do that. Conceptually, this task is not part of the TrustManager's responsibility; it's probably better done after connection is established, by examining peer certificate.

Oddly though, there is no public API for doing that. If you don't mind relying on sun.* package, you can use sun.security.util.HostnameChecker.match(String expectedName, X509Certificate cert).

I am also working on related issues, and I'll publish my API in a few days.

Wildcard certificate - I just raised the same question to jdk dev, waiting for a resonse - http://mail.openjdk.java.net/pipermail/security-dev/2014-September/011153.html

ZhongYu
  • 19,446
  • 5
  • 33
  • 61
  • Hmm.. The protocol specification I'm implementing seems to imply the order of checking (hostname check and then certificate validation). So I think an implementation of `TrustManager` interface is the right place to do this check in my case. – predi Sep 03 '14 at 06:38
  • Anyways. The text you quoted is also present in the reference guide for java 1.7 and confirms my suspicions and therefore answers my question. I remembered reading it somewhere, but couldn't find it. – predi Sep 03 '14 at 06:44
  • The class you mentioned is probably a part of HTTPS HostnameVerifier, so the checks might be specific to HTTPS specs. I'm going to roll on my own, but it seldom hurts to see another implementation. Thanks. – predi Sep 03 '14 at 06:52
  • Note that this answer is not correct. Java 7 has actually introduced a way to perform this verification automatically, but the guide isn't necessarily clear about it (or hasn't been updated). – Bruno Sep 03 '14 at 08:23
  • Btw, the implementation in @Bruno's answer does the check **during** TLS negotiation. Doing the check after a successful connection has already been made seems less secure to me. No choice but to rely on a `TrustManager`... – predi Sep 03 '14 at 09:18
  • @predi It's fine as long as the client does not send data before verifying the host name. (I remember IE does that - it checks hostname after handshake is completed). – ZhongYu Sep 03 '14 at 17:04