Amazon AWS Cognito and Python Boto3 to establish AWS connection and upload file to Bucket

Question

I'm trying to use the AWS cognito service to authenticate and upload a file. I have been provided my regionType, identityPool, AWS account ID, and UnAuthRole. I also know the production and development bucket names.

I think I am setting the AWS Access Key and AWS Secret Key... I want to authenticate with cognito and use the results to allow me to do an bucket listing and later a file upload.

What am I doing wrong? How can I use the cognito id to establish an S3 connection?

Here is my code and the resulting error:

#!/usr/bin/python

import boto3
import boto
#boto.set_stream_logger('foo')
import json
client = boto3.client('cognito-identity','us-east-1')
resp =  client.get_id(AccountId='<ACCNTID>',IdentityPoolId='<IDPOOLID>')
print "\nIdentity ID: %s"%(resp['IdentityId'])
print "\nRequest ID: %s"%(resp['ResponseMetadata']['RequestId'])
resp = client.get_open_id_token(IdentityId=resp['IdentityId'])
token = resp['Token']
print "\nToken: %s"%(token)
print "\nIdentity ID: %s"%(resp['IdentityId'])
resp = client.get_credentials_for_identity(IdentityId=resp['IdentityId'])
secretKey = resp['Credentials']['SecretKey']
accessKey = resp['Credentials']['AccessKeyId']
print "\nSecretKey: %s"%(secretKey)
print "\nAccessKey ID: %s"%(accessKey)
print resp
conn = boto.connect_s3(aws_access_key_id=accessKey,aws_secret_access_key=secretKey,debug=0)
print "\nConnection: %s"%(conn)
for bucket in conn.get_all_buckets():
    print bucket.name

Error:

   Traceback (most recent call last):
  File "./test.py", line 32, in <module>
    for bucket in conn.get_all_buckets():
  File "/usr/local/lib/python2.7/dist-packages/boto/s3/connection.py", line 440, in get_all_buckets
    response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId>ASIAILXMPZEMJAVZN7TQ</AWSAccessKeyId><RequestId>10631ACFF95610DD</RequestId><HostId>PGWDRBmhLjjv8Ast8v6kVHOG3xR8erJRV2ob3/2RmqHXwrg8HCZV578YsNLaoL24Hknr+nh033U=</HostId></Error>

This corresponding iOS code works fine:

AWSCognitoCredentialsProvider *credentialsProvider =
[AWSCognitoCredentialsProvider credentialsWithRegionType:awsCognitoRegionType
                                               accountId:awsAccountId
                                          identityPoolId:awsCognitoIdentityPool
                                           unauthRoleArn:unauthRoleArn
                                                  authRoleArn:nil];

AWSServiceConfiguration *configuration = [AWSServiceConfiguration configurationWithRegion:awsCognitoRegionType
                                                                      credentialsProvider:credentialsProvider];

....

AWSS3TransferManagerUploadRequest *uploadRequest = [AWSS3TransferManagerUploadRequest new];
uploadRequest.bucket = [ELEEnvironment currentEnvironment].userDataS3Bucket;
uploadRequest.key = key;
uploadRequest.body = uploadFileURL;
[[self uploadTask:uploadRequest] continueWithExecutor:[BFExecutor mainThreadExecutor]...

Thanks for any help!

score 2 · Accepted Answer · answered Mar 12 '15 at 17:03

This question is really invalid because the authentication was failing not on creating a session but when trying to list the buckets.

Uploading and downloading from a specific bucket works fine with the above code but not the listing of all buckets.

# Upload a new file
data = open('test.jpg', 'rb')
s3.Bucket('mybucket').put_object(Key='test.jpg', Body=data)

# S3 Object
obj = s3.Object(bucket_name='mybucket', key='test.jpg')
response = obj.get()
data = response['Body'].read()
print len(data)

score 1 · Answer 2 · answered Feb 09 '17 at 08:52

PhilBot, I don't know why your original code sample connects to s3 using boto (as opposed to boto3). The code connects to cognito using boto3. As of now, boto3 is stable and there's probably not much reason to use boto anymore. (Maybe when you originally posted your question, boto3 was not as stable as it is today.)

When I tried using your code to connect to kinesis with boto3, it didn't work -- I had to pass response["Credentials"]["SessionToken"] as the aws_session_token to the client() function.

score 0 · Answer 3 · answered Mar 11 '15 at 15:51

0

This is your error:

File "./test.py", line 32, in <module>
bucket = conn.get_bucket("elektradevbucket")

This is your part of the code that references the bucket:

bucket = conn.get_bucket("testbucket")
'''
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
    print(bucket.name)
s3.Bucket('testbucket')

Are you sure you are running or calling the correct script?

Best, -Iulian

answered Mar 11 '15 at 15:51

Iulian

319
4
13

Hi lulian, I forgot to rename that in my post. The bucket names exist and match my code but I think that the 403 is because I am providing credentials incorrectly. Do you see anything wrong with my auth flow? Where should I use the token? – PhilBot Mar 11 '15 at 15:53
I have updated my post - I get a 403 when trying to list all buckets. So the connect_s3 is not successful even though every call up to that point is successful. – PhilBot Mar 11 '15 at 16:00

Amazon AWS Cognito and Python Boto3 to establish AWS connection and upload file to Bucket

3 Answers3

Linked