Access denied when trying to do AWS s3 ls using AWS cli

Question

I launched an ec2 instance and created a role with a full S3 access policy for the instance. I installed awscli on it and configured my user's access key. My user has admin access and full S3 access policy too. I can see the buckets in the aws console but when I try to run aws s3 ls on the instance it returned An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied.

What else I need to do to add permission to the role or my user properly to be able to list and sync object between S3 and the instance?

Run `aws sts get-caller-identity` to verify that you're using the role/credentials that you think you are. — guest, Apr 10 '19 at 19:06
@guest I ran `aws sts get-caller-identity` and it showed my user information. I attached the AmazonS3FullAccess policy to my user and I thought this is sufficient to operate. @kenlukas I added the role with AmazonS3FullAccess policy to the instance. Is it what you mean by the server? — bliu, Apr 10 '19 at 19:33
Is the S3 bucket that you're trying to access in the same AWS account as your user? — guest, Apr 10 '19 at 19:38
Then without looking over your shoulder I don't know what to tell you. If you've attached the policy to your user or the role, you should be able to access the bucket. — guest, Apr 10 '19 at 19:59
When you ran `aws sts get-caller-identity` you mentioned that it showed your user information. Did you configure your credentials to be default profile? Can you confirm that AmazonS3FullAccess policy is attached to the IAM user? — krishna_mee2004, Apr 10 '19 at 20:10
I had this just now. Created a new user and added AmazonS3FullAccess permission without setting anything else. I got `Access Denied` on any operation. Apparantly I had to wait for 10 minutes before it was working. After that I could do anything using `aws cli s3` But I did change file `~/.aws/credentials`, updated the previous default keys with the new keys. Maybe the old default settings were cached? — A.W., Aug 13 '21 at 09:07

Varun Tandon · Answer 1 · 2022-05-03T04:47:40.353

4

I ran into this issue as well.

I ran aws sts get-caller-identity and noticed that the ARN did not match what I was expecting. It turns out if you have AWS configurations set in your bash_profile or bashrc, the awscli will default to using these instead.

I changed the enviornment variables in bash_profile and bashrc to the proper keys and everything started working.

edited May 03 '22 at 04:47

answered Aug 01 '19 at 21:59

Varun Tandon

51
6

score 1 · Answer 2 · answered Apr 11 '19 at 03:47

1

Turns out I forgot I had to do mfa to get access token to be able to operate in S3. Thank you for everyone response.

answered Apr 11 '19 at 03:47

bliu

153
2
10

i tried the same by creating profile with access token, secret access and session token but still getting the same error. Is there any other way we can debug? – Vishnu Sep 04 '20 at 02:27

score 0 · Answer 3 · answered Apr 10 '19 at 22:14

There appears to be confusion about when to use IAM Users and IAM Roles.

When using an Amazon EC2 instance, the best method to grant permissions is:

Create an IAM Role and attach policies to grant the desired permissions
Associate the IAM Role with the Amazon EC2 instance. This can be done at launch time, or afterwards (Actions/Instance Settings/Attach IAM Role).
Any application running on the EC2 instance (including the AWS CLI) will now automatically receive credentials. Do not run aws configure.

If you are wanting to grant permissions to your own (non-EC2) computer, then:

Create an IAM User (or use your existing one) and attach policies to grant the desired permissions
On the computer, run aws configure and enter the Access Key and Secret Key associated with the IAM User. This will store the credentials in ~/.aws/credentials.
Any application running on this computer will then use credentials from the local credentials file

score 0 · Answer 4 · answered Oct 18 '19 at 06:04

0

Create a IAM user with permission.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::bucketName/*"
        }
    ]
}

Save Access key ID & Secret access key.

sudo apt install awscli
aws configure
AWS Access Key ID [None]: AKIAxxxxxxxxxxxZI4
AWS Secret Access Key [None]: 8Bxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx8
Default region name [None]: region (ex. us-east-2)
Default output format [None]: json

aws s3 ls s3://s3testingankit1/

answered Oct 18 '19 at 06:04

Ankit Kumar Rajpoot

5,188
2
38
32

i tried your answer. Even i get session token and created profile and tried accessing with that but still same issue. – Vishnu Sep 04 '20 at 02:25
For try, You can give all full s3 permission to your IAM User. – Ankit Kumar Rajpoot Sep 04 '20 at 05:22

score 0 · Answer 5 · answered May 28 '20 at 00:59

This problem can occurs not only from the CLI but also when executing S3 API for example.

The reason for this error can come from wrong configuration of the access permissions to the bucket.

For example with the setup below you're giving a full privileges to perform actions on the bucket's internal objects, BUT not specifying any action on the bucket itself:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "arn:aws:s3:::<name-of-bucket>/*"
            ]
        }
    ]
}

This will lead to the mentioned

... (AccessDenied) when calling the ListBuckets ...

error.

In order to fix this you should allow application to access the bucket (1st statement item) and to edit all objects inside the bucket (2nd statement item):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<name-of-bucket>"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::<name-of-bucket>/*"
            ]
        }
    ]
}

There are shorter configurations that might solve the problem, but the one specified above tries also to keep fine grained security permissions.

score 0 · Answer 6 · answered Feb 03 '22 at 17:28

I ran into this yesterday running a script I ran successfully in September 2021.

TL;DR: add --profile your.profile.name to the end of the command

I have multiple profiles on the login I was using. I think something in the aws environment changed, or perhaps I had done something that was able to bypass this before. Back in September I set the profile with

aws configure set region us-west-2 --profile my.profile.name

But yesterday, after the failure, I saw that aws sts get-caller-identity was returning a different identity. After some documentation search I found the additional method for specifying the profile, and operations like:

aws s3 cp myfile s3://my-s3-bucket --profile my.profile.name

all worked

score 0 · Answer 7 · answered Feb 13 '23 at 09:52

I have an Windows machine with CyberDuck from which I was able to access a destination bucket, but when trying to access the bucket from a Linux machine with aws command, I got "An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied".

I then executed same command "aws s3 ls" from a command line interface on the Windows machine and it worked just fine. It looks like there is some security restriction on the AWS server for the machine/IP.

Access denied when trying to do AWS s3 ls using AWS cli

7 Answers7

Linked