3

Looking for some guidance with regards to uploading files into AWS S3 bucket via a python script and an IAM role. I am able to upload files using BOTO3 and an aws_access_key_id & aws_secret_access_key for other scripts.

However, I have now been given an IAM role to login to a certain account. I have no issue using AWS CLI to authenticate and query the S3 data so I do believe that my .aws/credential and .aws/config files are correct. However I am not sure how to use the ARN value within my python code.

This is what I have put together so far, but get a variety of errors which all lead to denied access:

session = boto3.Session(profile_name='randomName')
session.client('sts').get_caller_identity()

assumed_role_session = boto3.Session(profile_name='randomNameAccount')
print(assumed_role_session.client('sts').get_caller_identity())

credentials = session.get_credentials()
aws_access_key_id = credentials.access_key
aws_secret_access_key = credentials.secret_key

s3 = boto3.client('s3',
                  aws_access_key_id=aws_access_key_id,
                  aws_secret_access_key=aws_secret_access_key)

bucket_name = 'bucketName'

This is a sample of what my credential and config files looks like as a referal.

.aws/config file:

[profile randomNameAccount]
role_arn = arn:aws:iam::12345678910:role/roleName
source_profile = randomName

aws/credentials file:

[randomName]
aws_access_key_id = 12345678910
aws_secret_access_key = 1234567-abcdefghijk

My question is help around the python code to be able to authenticate against AWS and navigate around a S3 bucket using an IAM role and then upload files when I call an upload function.

Thank you in advance.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
MdM
  • 81
  • 2
  • 9
  • There is no logging into a bucket, you can request information from a bucket (which objects are stored, etc) and if you have appropriate permissions, that content is sent back to you. For your question about how to use roles, your IAM user is assuming the role, and the role has specific permissions associated. See this posting which may help clarify: https://stackoverflow.com/questions/48274044/how-to-generate-access-key-secret-key-for-aws-roles – Lucas Roberts Mar 04 '22 at 22:36
  • Thank you Lucas. I have read through it. I have no issue using the AWS CLI to navigate around the bucket. I just need help with the Python code around authentication. – MdM Mar 04 '22 at 23:18
  • Do you get an error if you try to access an object in the bucket? It looks like this code uses the boto3 s3 client, so does something like: `s3.download_file('bucket-name', 'object-name', '/file/path/local-fs/download/location')` give you a client error? If so, can you add it to the post? – Lucas Roberts Mar 05 '22 at 03:22
  • also, take a look at the ordering rules for credentials here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials you might be overriding things by explicitly putting the key & secret into the client constructor in the line`s3 = boto3.client('s3', ...` – Lucas Roberts Mar 05 '22 at 03:24

1 Answers1

4

You should create an entry for the IAM Role in ~/.aws/credentials that refers to a set of IAM User credentials that have permission to assume the role:

[my-user]
aws_access_key_id = AKIAxxx
aws_secret_access_key = xxx

[my-role]
source_profile = my-user
role_arn = arn:aws:iam::123456789012:role/the-role

Add an entry to ~/.aws/config to provide a default region:

[profile my-role]
region = ap-southeast-2

Then you can assume the IAM Role with this code:

import boto3

# Create a session by assuming the role in the named profile
session = boto3.Session(profile_name='my-role')

# Use the session to access resources via the role
s3_client = session.client('s3')
response = s3_client.list_objects(Bucket=...)
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470