1

I'm using the following code (excerpt) in my backend to create a presigned URL to an mp3 file within an AWS S3 bucket:

const s3Client = new S3Client({
    credentials: {
        accessKeyId: "AAAAAAAAAAAAAAAAAAAA",
        secretAccessKey: "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
    }
});

const getObjectCommand = new GetObjectCommand({
    Bucket: "my-bucket-name",
    Key: "file-to-be-used"
});

// Generate a signed URL for the GET request
const url = await getSignedUrl(s3Client, getObjectCommand, { expiresIn: 300 }).then(data => {
    ...
    do some stuff and return the url
    ...
});

Postman

Calling this code in Postman works as expected and the presigned URL returned from my backend works immediately after getting the response in Postman (e.g. by just clicking on it).

Browser

In my browser, however, I get 403 - Access forbidden errors when using the URL, e.g. for either setting it as the source for an HTMLAudioElement or getting it via a fetch request.

Confusing behaviour (in the browser)

When I wait for about 15 seconds before using/accessing the returned URL, then it works as expected (just clicking on it opens in the browser, setting it as source for the HTMLAudioElement also works).

Screen Recording

The screen recording from the browser shows how the request returns a presigned URL. It also shows the 403 errors (one when trying to set the URL as source for an HTMLAudioElement, the second when trying to fetch the presigned URL from S3).

You can also see that clicking on the URL at first leads to 403 errors (in the new browser tab that opens) twice. When clicking on the same URL for the third time it works (also works when just waiting for about 15 seconds as described above). Browser recording

  • I do not get any CORS related errors (and I've set up CORS with AllowedHeaders and AllowedOrigins set to * each)
  • In Postman the returned URL works immediately
  • The URL I try to access is exactly the same (as I'm using my own backend) for Postman and within the browser

Any ideas what might be wrong?

Update

As suggested in the comments, I've tried with another bucket in the standard region (us-east-1), same results:

Browser recording with bucket in the standard region

The request and headers in Postman

As it was sent by Postman

https://presigned-url-test-tim.s3.us-east-1.amazonaws.com/4f65f31b-7d63-4be0-a289-1c3a8e056ea4.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIASAQ3L4XYIZI4YD73%2F20230311%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230311T220846Z&X-Amz-Expires=300&X-Amz-Signature=b3c081018ea7e9f7979a65267f66bfdcf2f185d2f824bd86fa9ae9141e8ba627&X-Amz-SignedHeaders=host&x-id=GetObject and again for better readability

https://presigned-url-test-tim.s3.us-east-1.amazonaws.com/4f65f31b-7d63-4be0-a289-1c3a8e056ea4.mp3?
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&
X-Amz-Credential=AKIASAQ3L4XYIZI4YD73%2F20230311%2Fus-east-1%2Fs3%2Faws4_request&
X-Amz-Date=20230311T220846Z&
X-Amz-Expires=300&
X-Amz-Signature=b3c081018ea7e9f7979a65267f66bfdcf2f185d2f824bd86fa9ae9141e8ba627&
X-Amz-SignedHeaders=host&
x-id=GetObject

Additionally, Postman shows the following info under Request headers in the Console log:

User-Agent: PostmanRuntime/7.31.1
Accept: */*
Postman-Token: 4364aac9-1334-4434-b9a9-0b792dbde7b5
Host: presigned-url-test-tim.s3.us-east-1.amazonaws.com
Accept-Encoding: gzip, deflate, br
Connection: keep-alive

The request headers in the browser

As it was sent by the browser

https://presigned-url-test-tim.s3.us-east-1.amazonaws.com/4f65f31b-7d63-4be0-a289-1c3a8e056ea4.mp3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIASAQ3L4XYIZI4YD73/20230311/us-east-1/s3/aws4_request&X-Amz-Date=20230311T220557Z&X-Amz-Expires=300&X-Amz-Signature=e58bdbb1af3d4d47e1ec7c2aeaab3f77dd3a1fd7987831f1d0e6367a359637ef&X-Amz-SignedHeaders=host&x-id=GetObject and again for better readability

https://presigned-url-test-tim.s3.us-east-1.amazonaws.com/4f65f31b-7d63-4be0-a289-1c3a8e056ea4.mp3?
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&
X-Amz-Credential=AKIASAQ3L4XYIZI4YD73/20230311/us-east-1/s3/aws4_request&
X-Amz-Date=20230311T220557Z&
X-Amz-Expires=300&
X-Amz-Signature=e58bdbb1af3d4d47e1ec7c2aeaab3f77dd3a1fd7987831f1d0e6367a359637ef&
X-Amz-SignedHeaders=host&
x-id=GetObject
Gorgsenegger
  • 7,356
  • 4
  • 51
  • 89
  • If you test this in a private/incognito browser session, does it work? Not clear why the browser might somehow be caching a response for a URL that it's never seen before. – jarmod Mar 10 '23 at 23:38
  • @jarmod Same behaviour :-( Tried in Firefox, Edge and Chrome and exactly the same - after about 15 seconds the link becomes accessible. – Gorgsenegger Mar 11 '23 at 07:02
  • ...and also happens in the same way with curl. Looking at the `GET` request in Postman also doesn't give me any clues - the very same parameters in the same order and the same data (except in signature and date as the requests were made at slightly different times) as in the browsers. – Gorgsenegger Mar 11 '23 at 07:17
  • Hard to see why postman would work but curl would not. It's also not like there's anything going on in the AWS backend wrt eventual consistency or with S3 because the URL signing doesn't interact with S3 at all. Interesting problem. Does a capture of the postman invocation of the URL show different HTTP headers than the curl/browser invocation? Does using a non-regional bucket (using us-east-1 so endpoint is bucket.s3.amazonaws.com) behave differently? – jarmod Mar 11 '23 at 13:31
  • Changing the region of the bucket didn't improve anything. I also updated the question with the headers from Postman and the browser requests. I'll look into it with another system tomorrow, might (for whatever reason) be related to my specific machine. Still confused as you rightly said - presigning the URL doesn't call anything on the AWS side... – Gorgsenegger Mar 11 '23 at 22:16
  • Same behaviour on a different system - Postman works, browser shows access denied for about 15 seconds when trying to query the same URL, afterwards it works. Comparing the log entries from Cloudtrail show that the only differences are a) userAgent b) `x-amz-id-2` and `requestID` (as is expected) and `eventID` and c) the error message itself (in the failing request) – Gorgsenegger Mar 13 '23 at 13:38

1 Answers1

2

The solution is very easy and also obvious when thinking about it. I didn't realise that the problem was that I wrote the file to the S3 bucket using PollyClient and StartSpeechSynthesisTaskCommand (just was focused on the 403 - Access Denied errors).

When the command returns, the task is not completed and usually in the status scheduled, so the file isn't there yet, even though the (future) file name is already returned in the response.

Immediately generating a presigned URL based on that filename always works as this doesn't perform any calls to AWS, the URL is generated locally.

Trying to access the URL while the task is still not completed, however, results in 403 - Access Denied errors. Unfortunately this is somewhat misleading and led me to investigate in different directions before realising my mistake.

As soon as the task completes (and that usually takes about 15 seconds in my experience) the presigned URL works as expected.

Gorgsenegger
  • 7,356
  • 4
  • 51
  • 89
  • Glad you resolved the problem. The Postman info was a bit of a red herring. – jarmod Mar 15 '23 at 20:33
  • @jarmod, yes, that wasn't helpful :-/ I also would have liked to get a 404 from AWS instead of a 403, that might've sent me into the right direction sooner. – Gorgsenegger Mar 16 '23 at 08:55
  • You'd get 404 if the signing credentials had ListBucket permission (see [here](https://stackoverflow.com/questions/19037664/how-do-i-have-an-s3-bucket-return-404-instead-of-403-for-a-key-that-does-not-e)). – jarmod Mar 16 '23 at 13:22
  • Ah... yes, okay. The downside of keeping the permissions restrictive :-) Thank you for the link/explanation. – Gorgsenegger Mar 16 '23 at 20:09