How do I put object to amazon s3 using presigned url?

Question

I am trying to use signed url to upload images to s3 bucket. Following is my bucket policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::12345678:user/myuser",
                    "arn:aws:iam::12345678:root"
                ]
            },
        "Action": [
                "s3:List*",
                "s3:Put*",
                "s3:Get*"
            ],
            "Resource": [
                "arn:aws:s3:::myBucket",
                "arn:aws:s3:::myBucket/*"
            ]
        }
    ]
}

I am generating the signed url from the server as follows:

var aws = require('aws-sdk');
aws.config = {
    accessKeyId: myAccessKeyId,
    secretAccessKey: mySecretAccessKey
};

var s3 = new aws.s3();
s3.getSignedUrl('putObject', {
    Bucket: 'myBucket',
    Expires: 60*60,
    key: 'myKey'
}, function (err, url) {
    console.log(url);
});

I get the url. But when I try to put an object I get the following error:

<Error>
    <Code>AccessDenied</Code>
    <Message>Access Denied</Message>
    <RequestId>FXXXXXXXXX</RequestId>
    <HostId>fXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</HostId>
</Error>

Update 1

Here is myuser's policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
            "AWS": [
                "arn:aws:iam::2xxxxxxxxxxx:user/myuser",
                "arn:aws:iam::2xxxxxxxxxxx:root"
            ]
        },
        "Action": [
            "s3:*"
        ],
        "Resource": [
            "arn:aws:s3:::myBucket",
            "arn:aws:s3:::myBucket/*"
        ]
    }
  ]
}

Update 2 I can upload only when following option is set. I dont understand whats the use of bucket policy if only the manual selection of permission work.

Update 3

The following code works. Now the only problem is the signed url

 #!/bin/bash

 file="$1"

 bucket="mybucket"
 resource="/${bucket}/${file}"
 contentType="image/png"
 dateValue=`date -R`
 stringToSign="PUT\n\n${contentType}\n${dateValue}\n${resource}"
 s3Key="AKxxxxxxxxxxxxxxxxx"
 s3Secret="/Wuxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
 signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret}     -binary | base64`
 curl -X PUT -T "${file}" \
   -H "Host: ${bucket}.s3.amazonaws.com" \
   -H "Date: ${dateValue}" \
   -H "Content-Type: ${contentType}" \
   -H "Authorization: AWS ${s3Key}:${signature}" \
   https://${bucket}.s3.amazonaws.com/${file}

I assume myAccessKeyId and mySecretAccessKey both belong to myuser? — Brooks, Mar 07 '16 at 21:49
You are uploading a file named "myKey" when you generate the url for "myKey", right? — Volkan Paksoy, Mar 09 '16 at 15:37
@VolkanPaksoy not necessarily. the file is renamed to myKey after uploading to s3. The problem is not the filename here. the problem is permission. The file uploads when I set the permission to everyone, and access is denied when I dont; regardless of my bucket or user policy — Pravin, Mar 09 '16 at 16:22
I see. I don't think you'd need "Principal" is user's policy, it's attached to that user anyway. I'd try to attach that user a full S3 access policy selected from the list and try to upload a file with his keys using a tool like CloudBerry to avoid any possible oversight with the code or manual policies. Once you can manually upload files I'd try the presigned urls. — Volkan Paksoy, Mar 09 '16 at 16:54
@VolkanPaksoy okay. let me try those first and get to you. Thanks. :) — Pravin, Mar 09 '16 at 17:02
@VolkanPaksoy Please check my new update. The final put code is working. So the policy is correct. I am completely unaware what the problem is. — Pravin, Mar 09 '16 at 22:34

score 25 · Accepted Answer · edited Oct 25 '22 at 12:27

I managed to succesfully upload a file by using your code.

Here are the steps I followed:

Created a new bucket and a new IAM user

Set IAM user's policy as below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1418647210000",
            "Effect": "Allow",
            "Action": [
                "s3:Put*"
            ],
            "Resource": [
                "arn:aws:s3:::myBucket/*"
            ]
        }
    ]
}

Did NOT create a bucket policy

Used your code to generate the pre-signed URL:

var aws = require('aws-sdk');
aws.config = {
    accessKeyId: myAccessKeyId,
    secretAccessKey: mySecretAccessKey
};

var s3 = new aws.s3();
s3.getSignedUrl('putObject', {
    Bucket: 'myBucket',
    Expires: 60*60,
    Key: 'myKey',
    ContentType: 'image/jpeg',
}, function (err, url) {
    console.log(url);
});

Copied the URL on the screen and used curl to test the upload as below:

curl.exe -k -X PUT -T "someFile" "https://myBucket.s3.amazonaws.com/myKey?AWSAccessKeyId=ACCESS_KEY_ID&Expires=1457632663&Signature=Dhgp40j84yfjBS5v5qSNE4Q6l6U%3D"

In my case it generally took 5-10 seconds for the policy changes to take effect so if it fails the first time make sure to keep sending it for a while.

Please note: if you get CORS errors, ensure you have provided a ContentType argument to the s3.getSignedUrl call. As Hugo Mallet discusses below, "When you upload, your browser will add the content-type to the request headers. [Therefore, if you do not provide the MediaType server-side, there will be] a difference between the executed request and the signature you obtained with getSignerUrl. Of course you have to set the correct content-type depending on the file you want to upload."

Hello, Can you please tell what myKey is in the `getSignedUrl` function. — formatkaka, Jan 24 '17 at 09:59
@Siddhant That's basically the name of the object on S3. Check out the parameters descriptions here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putObject-property — Volkan Paksoy, Jan 24 '17 at 11:37
may I suggest: - var s3 = new aws.S3({ signatureVersion: 'v4', region: 'eu-central-1' }) - Key: 'images/upload/noek.jpg' — Paul, Feb 28 '19 at 11:40
I have spent almost 3.5 hours on this. And IAM user with the apt-permission set is the right way to get it done. — Satya Kalluri, May 06 '21 at 13:49

score 8 · Answer 2 · answered Mar 30 '18 at 13:49

8

It may help you too :) Add a ContentType property :

s3.getSignedUrl('putObject', {
    Bucket: 'myBucket',
    Expires: 60*60,
    Key: 'myKey',
    ContentType: 'image/jpeg',
}, function (err, url) {
   console.log(url);
});

answered Mar 30 '18 at 13:49

Hugo Mallet

533
5
8

1

I have no ideia why, but I was getting signature errors until I added this! – nelsonec87 Dec 05 '19 at 18:52
1

When you upload, your browser will add the content-type to the request headers. This will create a difference between the executed request and the signature you obtained with getSignerUrl. Of course you have to set the correct content-type depending on the file you want to upload. – Hugo Mallet Dec 11 '19 at 16:02

score 3 · Answer 3 · answered Mar 09 '16 at 17:06

3

In your IAM console, click Users
On the right list, choose the IAM user you used(should be 'myuser')
Choose Permissions on the sub tabs
Click Attach Policy and choose AmazonS3FullAccess

The final page will be like this.

You may also check Security Credentials sub tab, your accessKeyId should be on the list. The secretAccessKey just can not get again.

answered Mar 09 '16 at 17:06

hankchiutw

1,546
1
12
15

1

It has the full administrator access for now. I can upload from the s3 console. The problem is I cannot upload using signed url. – Pravin Mar 09 '16 at 17:13

Ivy · Answer 4 · 2020-05-08T19:39:14.057

2

You've correctly set up the permissions on the bucket, to allow access from the user.

But you also need to edit the policy of the user, to allow the user to access the S3 service.

Edit the IAM policy of the user whose credentials you're using to generate the self-signed URL. This policy allows a user full administrative access to all S3 buckets in the account:

{
  "Statement": [
    {
      "Sid": "AllowAllS3Access",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

edited May 08 '20 at 19:39

answered Mar 08 '16 at 15:10

Ivy

3,393
11
33
46

i still dont understand man. what is going on? it still gives access denied error. – Pravin Mar 08 '16 at 15:40
3

This grants the user all rights to all S3 buckets. Pretty open and unsafe. – Samuel Neff Sep 25 '17 at 02:28
This grants the user permission to do anything in the S3 service. It doesn't automatically give them access to buckets, unless the bucket policy allows them. – Ivy Nov 30 '21 at 22:43

score 1 · Answer 5 · answered Mar 04 '21 at 06:22

Here is the complete code for generating pre-signed URL for any type of file in S3.

If you want you can include expiration time using Expire parameter in parameter.
The below code will upload any type of file like excel(xlsx, pdf, jpeg)

    const AWS = require('aws-sdk');
    const fs = require('fs');
    const axios = require('axios');
    const s3 = new AWS.S3();
    const filePath = 'C:/Users/siva.cheedella/Downloads/invoice.pdf';
    
    
    var params = {
        Bucket: 'testing-presigned-url-dev',
        Key: 'dummy.pdf',
        "ContentType": "application/octet-stream"
    };
    
    
    s3.getSignedUrl('putObject', params, function (err, url) {
        console.log('The URL is', url);
    
        fs.writeFileSync("./url.txt", url);
    
    
        axios({
            method: "put",
            url,
            data: fs.readFileSync(filePath),
            headers: {
                "Content-Type": "application/octet-stream"
            }
        })
            .then((result) => {
                console.log('result', result);
    
            }).catch((err) => {
                console.log('err', err);
    
            });
    
    });

How do I put object to amazon s3 using presigned url?

5 Answers5

Linked