aws credentials working in CLI, not in boto3

Asked Feb 09 '22 at 20:02

Active Feb 09 '22 at 20:02

Viewed 22 times

I have set up my own aws profile on a shared vm:

[profile me]
aws_access_key_id = xxxx
aws_secret_access_key = yyyy
region = zzzz

and if I try to access resources from s3 by the command line, everything works:

> aws s3 cp s3:/bucket/prefix/resource_name . --profile me
download: s3:/bucket/prefix/resource_name to ./resource_name

but if I try to access the same resources through Python, I get 403 errors:

session = boto3.session.Session(profile_name='me')
s3 = session.resource('s3')
df = pd.read_csv('s3://bucket/prefix/resource_name')

>> botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

any thoughts what's causing the different behavior?

asked Feb 09 '22 at 20:02

DSJ529

What is the relationship between `pd` and `s3`? I can't see a link. – John Rotenstein Feb 09 '22 at 21:17
You're not providing the S3 credentials to pd. You need to specify the credentials with something like `storage_options={'profile': "me"}` – Anon Coward Feb 09 '22 at 21:18

aws credentials working in CLI, not in boto3

0 Answers0