1

I'm trying to understand a bit the following situation. I have set $HOME/.aws/credentials as follows

[aws-users-andres]
aws_access_key_id = accesskey
aws_secret_access_key = UJPYNsecretkey

this credentials are refreshed periodically using some scripts and MFA. $AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN are set in the environment.

Then I also have $HOME/.aws/config file with a set of profiles that my company is using to access different resources:

[profile profile_name]
role_arn = arn:aws:iam::1464549XXXX:role/profile_name
credential_source = Environment

Now if I try to list the buckets:

aws s3 ls

The output is "An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied"

But running using the profile

aws s3 ls --profile profile_name

I'm able to list the buckets in our S3.

I'd like to list the buckets without specifying the profile or set the default behavior to use profile_name . I was recently tasked with a task to periodically sync some google cloud storage bucket with s3 resources, and was following the ideas from here Exporting data from Google Cloud Storage to Amazon S3 but the gsutil command fails ( Forbidden access to S3 )

I have tried to use AWS_PROFILE and AWS_DEFAULT_PROFILE env var set as profile_name but that doesn't seem to work.

I appreciate any help and guidance. I could write some ad hoc script to copy and upload the needed files from gcloud to s3 but I was hoping gsutil would be a better option.

Thanks in advance.

RolandDeschain
  • 483
  • 1
  • 4
  • 17
  • 1
    Assuming you are on Linux or Mac, you can create an alias like so: `alias s3list='aws s3 ls --profile profile_name'`. After that, you can type s3list to list the buckets. You can also create an environment variable like so: `export AWS_DEFAULT_PROFILE=profile_name`. See https://stackoverflow.com/questions/31012460 and https://phoenixnap.com/kb/linux-alias-command – zedfoxus Jul 04 '23 at 14:40
  • You can just modify the `config` file and name the profile `default` -- it will then be used by default! Use `[default]` (not `[profile default]`). – John Rotenstein Jul 05 '23 at 01:02

1 Answers1

1

Using inline environment variables:

AWS_ACCESS_KEY_ID="THISISMYKEY" \
AWS_SECRET_ACCESS_KEY="THIS_IS_SECRET_KEY" \
aws s3 ls

is the quickest but least secure way of doing this. It's insecure because running history | grep "AWS_ACCESS_KEY_ID" easily exposes the credentials used.

A better approach is to not use inline shell variables, avoid setting the credentials manually on those places and just utilise the built-in AWS CLI command for configuring profiles for your machine.

$ aws configure --profile allan
$ AWS Access Key ID [None]: THISISMYKEY
$ AWS Secret Access Key [None]: THIS_IS_SECRET_KEY
$ Default region name [None]: ap-southeast-1
$ Default output format [None]:

aws s3 ls --profile allan

If these two options don't work, consider seeking help from the person that issued your AWS credentials. The credentials may be stale, incorrect or lack permissions to query S3 buckets.

jarmod
  • 71,565
  • 16
  • 115
  • 122
Allan Chua
  • 9,305
  • 9
  • 41
  • 61