AWS signature v4 authentication succeeds for EU bucket but fails for US bucket?

Question

I recently implemented AWS Signature version 4 using the REST API. This is verified by an extensive regression test working perfectly.

The problem I'm experiencing is that the regression test succeeds when run against a bucket residing in the eu-central-1 region, but consistently fails with the Accessed Denied error message for buckets residing in us-east-1 or us-west-2.

Here are snippets from successful and failed attempts.

eu-central-1 : successful

HTTP request:

GET./
host:s3.eu-central-1.amazonaws.com.x-amz-content-sha256:e3b0...b855.x-amz-date:Wed, 25 May 2016 03:13:21 +0000
host;x-amz-content-sha256;x-amz-date.e3b0...b855

Signed string:

AWS4-HMAC-SHA256
Credential=AKIAJZN7UY6XHIZPWIKQ/20160525/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=cf5f...4dc8

Server response:

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult
    xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <Owner>
        <ID>100a...a575</ID>
    </Owner>
    <Buckets>
        <Bucket>
            . . .
        </Bucket>
    </Buckets>
</ListAllMyBucketsResult>

us-east-1 : failed

HTTP request:

GET./
host:s3.us-east-1.amazonaws.com.x-amz-content-sha256:e3b0...b855.x-amz-date:Wed, 25 May 2016 03:02:27 +0000
host;x-amz-content-sha256;x-amz-date.e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Signed string:

AWS4-HMAC-SHA256
Credential=AKIAJZN7UY6XHIZPWIKQ/20160525/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=01e97...4d00

Server response:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
    <RequestId>92EEF2A86ECA88EF</RequestId>
    <HostId>i3wTU6OzBrlX89xR4KnnezBx1Tb2IGN2wtgPJMRtKLjHxF/B6VdCQqPz1279J7e5</HostId>
</Error>

us-west-2 : failed

HTTP request:

GET./
host:s3.us-west-2.amazonaws.com.x-amz-content-sha256:e3b0...b855.x-amz-date:Wed, 25 May 2016 07:04:47 +0000
host;x-amz-content-sha256;x-amz-date.e3b0...b855

Signed string:

AWS4-HMAC-SHA256
Credential=AKIAJZN7UY6XHIZPWIKQ/20160525/us-west-2/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=cf70...36b9

Server response:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
    <RequestId>DB143DBF0F316EB8</RequestId>
    <HostId>5hWJ0AHM466QcT+BK4UaEFpqXFNaJFEuAPlN/ZZPBhL+NDYBoGaySRkXQ3BRdyfy9PBDuSb0oHA=</HostId>
</Error>

Attempts made to date include:

I found references (like here) where when using US Standard (i.e., us-east-1) the REST endpoint should not include "us-east-1". I have not yet found this written officially. I therefore created a us-west-2 bucket, in the hope that the REST endpoint needs to contain "us-west-2", but that also fails.
I searched on Google and StackOverflow for possible reasons for "Access Denied", which led me to adding a bucket policy that gives permissions to all -- to no avail.
The permissions of the EU and US accounts in the AWS console look the same, so no hint there, yet.
I added logging to the buckets in the hope of seeing a failure entry, but nothing is logged until authentication is completed.

Does anyone have an idea why AWS v4 authentication will consistently succeed for an eu-central-1 bucket, but equally fail for us-east-1 and us-east-2 buckets?

Did you ever manage to resolve this issue? I have a similar problem right now... Requests `us-east-1`(s3) fail consistently, but other Regions work fine. The error we get is the dreaded: `403: "The request signature we calculated does not match the signature you provided. Check your key and signing method.` — Silvano Brugnoni, Sep 05 '18 at 08:54

score 1 · Answer 1 · answered May 25 '16 at 23:02

Here's your issue.

For unknown reasons,¹ eu-central-1 is an oddball in S3. The REST endpoint works with two variations in hostname: bucket.s3.eu-central-1.amazonaws.com or bucket.s3-eu-central-1.amazonaws.com.

The difference is the dot or dash after s3.

All other regions (as of now) except us-east-1 and ap-northeast-2 (which is just like eu-central-1) work only with the dash after s3, e.g. bucket.s3-us-west-2.amazonaws.com... not with a dot.

And us-east-1 expects either bucket.s3.amazonaws.com or bucket.s3-external-1.amazonaws.com.

And finally, any region will work with just bucket.s3.amazonaws.com within a few minutes after the original creation of a bucket, because the DNS is integrated with the bucket location database and automatically routes requests to the right place, for each bucket.

But note that when you sign the requests, you always use the actual region name in the signing algorithm itself -- not the endpoint -- as you appear to already be doing.

http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

¹I'll speculate that this convention is actually the "new normal" for new regions -- it's more consistent with other AWS services. S3 is one of the oldest, so it makes sense that legacy design decisions are more likely to exist, as seems to be the case, here.

Your help is greatly appreciated -- use of "s3.", rather than "s3-", is found in the S3 Signature v4 documentation itself! After modifying from '.' to '-' in the host string, I find AWS is temporarily redirecting my bucket from us-east-1 to us-west-2. The server first returns "TemporaryRedirectPlease re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.", and then fails on "The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-2'". Will update soon. — Moshe Rubin, May 26 '16 at 06:59

AWS signature v4 authentication succeeds for EU bucket but fails for US bucket?

1 Answers1