AWS signed URL too long to shorten

Question

I am creating a signed URL with AWS so I can safely pass this URL to another API for temporary use. The signed URL points to a S3 resource. The problem is the other API does not accept such long links. Therefore I am trying to shorten it. I tried to use shorteners like goo.gl or bit.ly to no avail because the URL was too long for them. I even built my own private shortener with AWS (AWS url shortener) but it had the same problem: "The length of website redirect location cannot exceed 2,048 characters.".

I am creating the signed URLs in iOS (Swift) with AWSS3PreSignedURLBuilder.default().getPreSignedURL(preSignedURLRequest) while using AWS Cognito as an unauthorised user.

I have tried the following things to no avail:

Choose the shortest possible S3 bucket name with 3 characters
Shorten the filename as much as possible. I limited the file name to 10 characters plus file extension name (14 characters in total). Shorter file names are not viable for me because they should be unique to a certain extent.

But even with all these minor tweaks the signed URL returned by AWS is sometimes too long. Especially the token parameter (X-Amz-Security-Token) seems to be really long. With my minor tweaks I sometimes get URLs shorter than 2,048 characters but sometimes slightly longer. I would like to find a solution which guarantees me that the URL is not too long and can be shortened.

In my own private AWS URL shortener the following code snippet creates the S3 object which redirects to the actual long URL.

s3.putObject({
    Bucket: s3_bucket,
    Key: key_short,
    Body: "",
    WebsiteRedirectLocation: url_long,
    ContentType: "text/plain"
  },
  (err, data) => {
    if (err) {
      console.log(err);
      done("", err.message);
    } else {
      const ret_url = "https://" + cdn_prefix + "/" + id_short;
      console.log("Success, short_url = " + ret_url);
      done(ret_url, "");
    }
  });

The method returns with the following error

The length of website redirect location cannot exceed 2,048 characters.

The documentation of putObject for the header "x-amz-website-redirect-location" in the object meta states the following (see: put object documentation):

The length of the value is limited to 2 KB

How can I make sure that the initial AWS signed URL is not too long for the URL shorteners?

EDIT:

One of the problems I have identified is that I create the signed URL as an unauthenticated user in AWS Cognito. Therefore the signed URL includes this ridiculously long token as a parameter. I did not want to embed my accessKey and shortKey in the iOS App thats why I switched to AWS Cognito (see aws cognito). But currently there are no authorised users just unauthorised ones and I need to create the signed URL as an unauthorised AWS Cognito user. If I create the signed URL with with a regular credentials using accessKey and shortKey I get a much shorter URL. But for that I would have to embed my accessKey and shortKey in the iOS app which is not recommended.

@helloV I am creating the signed URLs with `AWSS3PreSignedURLBuilder.default().getPreSignedURL(preSignedURLRequest)` while using AWS Cognito as an unauthorised user. — christopher.online, Oct 21 '17 at 16:23
There's not too much that can be done if you are dependent on temporary credentials. *"Another API for temporary use"* is, I assume, not a an API that you control. Is that right? Would that API correctly follow a redirect from a shortener to the existing long URLs? I have an idea of another way to build a private shortener on AWS components that I *think* would be good for up to ~8,192 bytes, but would have to test it furher to see. What is the nature of the content of the signed URLs, and what kind of expiration times are you using in the signed URLs? — Michael - sqlbot, Oct 21 '17 at 19:43
@Michael-sqlbot You are correct in assuming that I cannot control the other API I am sending the shortened URL to. It will only accept quite short URLs. It will correctly follow the redirect from a shortener to an existing longer URL. I am currently using an expiration date of 3 days for the signed URL but will make it shorter in future, e.g. ball park of 1 hour. The signed URL should give the caller an opportunity to retrieve a file. The permission are also restricted so that only a certain IP can use those signed URLs. — christopher.online, Oct 21 '17 at 23:16

score 1 · Accepted Answer · answered Oct 24 '17 at 14:56

I solved the problem by creating an AWS lambda for creating a presigned URL and returning the presigned URL. The presigned URL allows the caller to access (getObject) the S3 resource. There are two options regarding this:

The role assigned to the AWS lambda has the S3 permission for getObject. The resulting presigned URL will have a much shorter token included than the presigned URL created with the temporary credentials issued by AWS Cognito in the iOS app.
Embed the access key and secret key of a role with the S3 permission for getObject directly into the AWS lambda which will give you an even shorter URL because there is no token included in the resulting presigned URL. (e.g. sample AWS code)

I call this lambda from within my iOS app as an unauthorised cognito user. After receiving the presigned URL from the AWS lambda I am able to shorten it because with this method the presigned URLs are much shorter.

score 0 · Answer 2 · answered Oct 21 '17 at 21:26

0

There is an older method of generating pre-signed URLs that produces a very short link, eg:

https://s3-ap-southeast-2.amazonaws.com/my-bucket/foo.png?AWSAccessKeyId=AKI123V12345RYTP123&Expires=1508620600&Signature=oB1/jca2JFXw5DbN7gBKEXkUQk8%3D

However, this pre-dates sigv4 so it does not work in the newer regions (Frankfurt onwards).

You can find sample code at:

mdwhatcott/s3_presign.py
S3 Generate Pre Signed Url
It can also be used to sign for uploads: Correct S3 Policy For Pre-Signed URLs

answered Oct 21 '17 at 21:26

John Rotenstein

241,921
22
380
470

unfortunately I am on EUCentral1. Could I create a lambda which uses a role which allows s3 getobject and create the signed url with the access and secret key of that role in the lambda function and call that lambda function from my iOS App using AWS cognito as an unauthorised user? Example code for the lambda I would call from within the iOS app would be e.g. https://stackoverflow.com/questions/38831829/nodejs-aws-sdk-s3-generate-presigned-url – christopher.online Oct 21 '17 at 23:29
The problem is that OP doesn't want to embed IAM credentials in the app, so you still have to accommodate the additional length of the `X-Amz-Security-Token`, which can be up to 4K bytes long. This would be shorter than a V4 URL, though. – Michael - sqlbot Oct 21 '17 at 23:31
@KaraBenNemsi that will result in a similar problem -- the Lambda function using a role would still generate a signed URL that would have an `X-Amz-Security-Token`. – Michael - sqlbot Oct 21 '17 at 23:34
I just tested with embedding the access key and the security key in the app an generating the presigned URL and it gave me a much shorter URL without the `X-Amz-Security-Token`. only `X-Amz-Algorithm` and `X-Amz-Credential` and `X-Amz-Date` and `X-Amz-Expires` and `X-Amz-SignedHeaders` and `X-Amz-Signature`. no token was contained – christopher.online Oct 21 '17 at 23:38
@Michael-sqlbot Could I not use that method (which seems to give me a shorter presigned URL) with specific role with access key and the security key embedded in lambda and create the presigned URL in the lambda function. and call that lambda function from the iOS side? – christopher.online Oct 21 '17 at 23:40

AWS signed URL too long to shorten

2 Answers2

Linked