PUTing files into S3 using Python requests

Question

I've got this URL that was generated using the generate_url(300, 'PUT', ...) method and I'm wanting to use the requests library to upload a file into it.

This is the code I've been using: requests.put(url, data=content, headers={'Content-Type': content_type}), I've also tried some variations on this but the error I get is always the same.

I get a 403 - SignatureDoesNotMatch error from S3 every time, what am I doing wrong?

http://stackoverflow.com/questions/10044151/how-to-generate-a-temporary-url-to-upload-file-to-amazon-s3-with-boto-library — varela, Jul 20 '12 at 14:09

score 1 · Answer 1 · answered Jul 26 '16 at 14:07

Using boto3, this is how to generate an upload url and to PUT some data in it:

session = boto3.Session(aws_access_key_id="XXX", aws_secret_access_key="XXX")
s3client = session.client('s3')
url = s3client.generate_presigned_url('put_object', Params={'Bucket': 'mybucket', 'Key': 'mykey'})

requests.put(url, data=content)

varela · Answer 2 · 2012-07-20T14:18:00.427

0

S3 requires authentication token if your bucket is not public write. Please check token.

I would suggest you to use boto directly.

bucket.new_key()
key.name = keyname
key.set_contents_from_filename(filename, {"Content-Type": content_type})
# optional if file public to read
bucket.set_acl('public-read', key.name)

Also please check did you add Content-Length header. It's required and take part in auth hash calculation.

edited Jul 20 '12 at 14:18

answered Jul 20 '12 at 14:02

varela

1,281
1
10
16

Could you please provide a source for that? AFAICT it's no different than `generate_url(300, 'GET', ...)` when it comes to permissions. – Noah McIlraith Jul 20 '12 at 14:12
I checked docs, you're right there is no difference for auth. Probably you missed headers for generate_url. – varela Jul 20 '12 at 14:15
That sounds about right, I didn't specify any, what headers are required? (Ah, I didn't see edit, Content-Length). – Noah McIlraith Jul 20 '12 at 14:20
I think Content-Length is, not sure about others – varela Jul 20 '12 at 14:22

score 0 · Answer 3 · answered Nov 23 '22 at 10:15

I ran into the same issue.

It seems the signature had a signed header "host" so it has to be included in the request you send as well.

s3_client = boto3.client(
    's3',
    region_name=<region>,
    config=Config(signature_version='s3v4')
)
kwargs = {
    "ClientMethod": "put_object",
    "Params": {
        "Bucket": <bucket>,
        "Key": <file key>,
    },
    "ExpiresIn": 3600,
    "HttpMethod": "PUT",
}
s3_client.generate_presigned_url(**kwargs)
with open(<my_file>, "rb") as f:
    data = f.read()
response = requests.put(url, data=data, headers={"host": "<bucket>.s3.amazonaws.com"})

PUTing files into S3 using Python requests

3 Answers3