10

Amazon "upgraded" the SSL security in its AWS Java SDK in the 1.3.21 version. This broke access any S3 buckets that have periods in their name when using Amazon's AWS Java API. I'm using version 1.3.21.1 which is current up to Oct/5/2012. I've provided some solutions in my answer below but I'm looking for additional work arounds to this issue.

If you are getting this error, you will see something like the following message in your exceptions/logs. In this example, the bucket name is foo.example.com.

INFO: Unable to execute HTTP request: hostname in certificate didn't match:
       <foo.example.com.s3.amazonaws.com> != <*.s3.amazonaws.com>
       OR <*.s3.amazonaws.com> OR <s3.amazonaws.com>
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:220)
at org.apache.http.conn.ssl.StrictHostnameVerifier.verify(StrictHostnameVerifier.java:61)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:149)
at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:130)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:390)

You can see documentation of this problem on the AWS S3 discussion forum:

https://forums.aws.amazon.com/thread.jspa?messageID=387508&#387508

Amazon's response to the problem is the following.

We should be able to fix this by using the older path style method of bucket addressing (instead of the newer virtual host style addressing) for buckets with this naming pattern. We'll get started on the fix and ensure that our internal integration tests have test cases for buckets names containing periods.

Any workaround or other solutions? Thanks for any feedback.

Gray
  • 115,027
  • 24
  • 293
  • 354

2 Answers2

9

Original: October 2012

Turns out that Amazon "upgraded" the SSL security on S3 in late September 2012. This broke access any S3 buckets that have periods in their name when using Amazon's AWS Java API.

This is inaccurate. S3's SSL wildcard matching has been the same as when S3 launched back in 2006. What's more likely is that the AWS Java SDK team enabled stricter validation of SSL certificates (good), but ended up breaking bucket names that have been running afoul of S3's SSL cert (bad).

The right answer is that you need to use path-style addressing instead of DNS-style addressing. That is the only secure way of working around the issue with the wildcard matching on the SSL certificate. Disabling the verification opens you up to Man-In-The-Middle attacks.

What I don't presently know is if the Java SDK provides this as a configurable option. If so, that's your answer. Otherwise, it sounds like the Java SDK team said "we'll add this feature, and then add integration tests to make sure it all works."

Update: October 2020

AWS has announced that path-style addressing is deprecated will be going away in the near-future. AWS’ advice is to use DNS-compatible bucket names, which means no periods (among a few other things). Certain newer features of S3 require DNS-compatible bucket names (e.g., accelerated transfer).

If you require a bucket name which contains periods (which will also be disallowed for new buckets in the near future), my best advice is to put a CloudFront distribution in front of it if you want to hit it over HTTPS.

Ryan Parman
  • 6,855
  • 1
  • 29
  • 43
  • +1 @Ryan. I had assumed it was a S3 certificate change. The bug was introduced in the SDK in version 1.3.21. – Gray Oct 06 '12 at 15:00
  • This provides the simple, yet effective way to make your query path-style... https://shlomoswidler.com/2009/08/amazon-s3-gotcha-using-virtual-host.html#commentform – Max Phillips Dec 02 '16 at 21:14
6

Amazon released version 1.3.22 which resolves this issue. I've verified that our code now works. To quote from their release notes:

Buckets whose name contains periods can now be correctly addressed again over HTTPS.

There are a couple of solutions that I can see, aside from waiting till Amazon releases a new API.

  1. Obviously you could roll back to 1.3.20 version of the AWS Java SDK. Unfortunately I needed some of the features in 1.3.21.

  2. You can replace the org.apache.http.conn.ssl.StrictHostnameVerifier in the classpath. This is a hack however which will remove all SSL checking for Apache http connections I think. Here's the code that worked for me: http://pastebin.com/bvFELdJE

  3. I ended up downloading and building my own package from the AWS source jar. I applied the following approximate patch to the HttpClientFactory source.

    ===================================================================
--- src/main/java/com/amazonaws/http/HttpClientFactory.java     (thirdparty/aws)      (revision 20105)
+++ src/main/java/com/amazonaws/http/HttpClientFactory.java     (thirdparty/aws)    (working copy)
@@ -93,7 +93,7 @@

                        SSLSocketFactory sf = new SSLSocketFactory(
                                SSLContext.getDefault(),
-                               SSLSocketFactory.STRICT_HOSTNAME_VERIFIER);
+                               SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);

  4. The right fix is to change from domain-name bucket handling to path based handling.

Btw, the following seems like it might work but it does not. The AWS client specifically requests the STRICT verifier and does not use the default one:

SSLSocketFactory.getSystemSocketFactory().setHostnameVerifier(
    SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
Gray
  • 115,027
  • 24
  • 293
  • 354
  • 4
    To be fair, if you request `https://foo.example.com.s3.amazonaws.com/...` in your URL, the `*.s3.amazonaws.com` doesn't cover that host name: the wildcard doesn't propagate across the dots according to [the specs](http://tools.ietf.org/html/rfc2818#section-3.1) ("**.a.com matches foo.a.com but not bar.foo.a.com*"). What Amazon could do is get a certificate for `*.*.*.s3.amazonaws.com`. – Bruno Oct 05 '12 at 23:06
  • Agreed @Bruno. What the question is talking however is using their Java API. Basically I can't do anything using their API with a bucket that has a period in its name. – Gray Oct 05 '12 at 23:08
  • Just reading [RFC 6125](http://tools.ietf.org/html/rfc6125#section-6.4.3) (best practice spec, but not really implemented yet). I'm not sure how to interpret whether `*.*.*.something` would be allowed with rule 1. The right thing (if you really need a custom handler) would indeed be to write your own verifier that expects the right pattern. The problem with the ALLOW_ALL_HOSTNAME_VERIFIER is that it allows MITM attacks. – Bruno Oct 05 '12 at 23:14
  • Amazon has a way to address the buckets _without_ affecting the hostname. That's how _they_ are going to fix it. I was looking for a short/mid term solution to issue. – Gray Oct 05 '12 at 23:18
  • 1
    for reference the 1.3.22 does not fix this for non-US buckets. Path-style addressing requires to change the endpoint to the region specific one using #setEndpoint so a little more work is needed setting the endpoint for the s3client instance in use – squiddle Oct 22 '12 at 12:09
  • As of AWS SDK version 2.16.5, we can't connect to 3rd party on-premise bucket with domain like this: uat.somesystem.s3.somedomain.co.th, where the cert is for *.s3.somedomain.co.th. If we try to connect to https://somesystem.s3.somedomain.co.th/uat, the server would respond with 403 Forbidden. – Chakrit W Feb 25 '21 at 03:03