Amazon S3 Error: The request signature we calculated does not match the signature you provided. Check your key and signing method

Question

I am trying to get an object from my bucket but I always get the "signature does not match" error. I got my signature generation function from AWS sample code so I am sure that this works. I also tested the upload function and it works. I am only having trouble with the get object function. I spent a huge amount of hours verifying all scenarios/answers in this post but nothing worked. So here I am seeking your help.

Am I missing something in the headers? Here is a sample canonical request that gets created:

GET
/index.html
    
host:<bucketname>.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20211127T120453Z
    
host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

EDIT1:

I tried the AWS sample code by compiling it using javac, then ran it, the get object function works fine. But when added into my Android project, it does not work. To make sure that I copy the sample code as is, I generated its jar file and included the resulting jar file in my project. Still, same issue. This is frustrating. Argg!

EDIT2: Minimal reproducible example

1. Download the sample code from the link above
2. Add the code into your Android project except for com.amazonaws.services.s3.sample.RunAllSamples class.
3. Add this call in one of your activities: GetS3ObjectSample.getS3Object(<bucketName>, <regionName>, <awsAccessKey>, <awsSecretKey>);

Answer 1

This is an interesting one, and basically it boils down to the Android implementation of HttpURLConnection:

com.squareup.okhttp.internal.huc.HttpURLConnectionImpl

behaving differently to the JVM provided implementation:

sun.net.www.protocol.http.HttpURLConnection

The relevant difference here, is that around line 323 the Android implementation checks both:

• whether the doOutput flag has been set, and
• whether the method field is "GET"

If both these are true, it "helpfully" changes the method to "POST", which means the signature, based on the canonical request which includes the request method, is no longer valid. This is why the other samples work without issue; as they are already using a request method besides "GET", it isn't changed.

In the AWS sample code, the doOutput field is set at:

com/amazonaws/services/s3/sample/util/HttpUtils.java:86

so what I would suggest going forward is to use the definition of:

com.amazonaws.services.s3.sample.GetS3ObjectSample.getS3Object(...)

as a guide to how to calculate the required Authorization header, but use your preferred sane HTTP client instead of the:

com.amazonaws.services.s3.sample.util.HttpUtils

class provided with the samples.