9

I have created an IAM user with access to only one bucket. I have tested the credentials and permissions through web and python boto. Its working fine.

Now I have requirement to use these credentials and download the private file from that bucket through curl. 

signature="$(echo -n "GET" | openssl sha1 -hmac "f/rHQ8yCvPthxxxxxxxXxxxx" -binary | base64)"
date="$(LC_ALL=C date -u +"%a, %d %b %Y %X %z")"

curl -H "Host: my-bucket.s3.amazonaws.com" -H "Date: $date" -H "Authorization: AWS 'XXXAJX2NY3QXXX35XXX':$signature" -H "Content-Type: 'text/plain'" https://my-bucket.s3.amazonaws.com/path/to_file.txt

but i am getting the following error:

InvalidAccessKeyIdThe AWS Access Key Id you provided does not exist in our records.

Please help, how do I download the file using curl ? Is there anything am I missing or its not possible through curl command?

Thanks!

Ahsan
  • 11,516
  • 12
  • 52
  • 79
  • 2
    I think this is what you need mate : http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html – nafas Feb 17 '15 at 12:45
  • 1
    Your key `'XXXAJX2NY3QXXX35XXX'` should not be enclosed in `'` single quotes here. – Michael - sqlbot Feb 18 '15 at 17:06
  • Thanks @Michael-sqlbot removing `'` changed the error message. Now stuck on Invalid Signature Provided error. I did all mentioned option in http://stackoverflow.com/questions/2777078/amazon-mws-request-signature-calculated-does-not-match-the-signature-provided Any help? – Ahsan Feb 19 '15 at 12:36
  • Your problem now no longer matches your question text, but since you don't have any answers posted yet, you should edit the question to reflect the actual problem you are now experiencing, bearing in mind that it seems like this could easily become a [chameleon question](http://meta.stackexchange.com/questions/43478/exit-strategies-for-chameleon-questions) which would not be good. `signature="$(echo -n "GET" | openssl ...` is missing a *lot* of necessary information. "GET" is not a sufficient "string to sign." Refer to the documentation for the parameters, then edit or delete this question. – Michael - sqlbot Feb 19 '15 at 17:41

1 Answers1

8

Following is the example on how you can download with s3 curl script,

#!/bin/sh 
file=path/to/file 
bucket=your-bucket 
resource="/${bucket}/${file}" 
contentType="application/x-compressed-tar" 
dateValue="`date +'%a, %d %b %Y %H:%M:%S %z'`" 
stringToSign="GET 
${contentType} 
${dateValue} 
${resource}" 
s3Key=xxxxxxxxxxxxxxxxxxxx 
s3Secret=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
signature=`/bin/echo -en "$stringToSign" | openssl sha1 -hmac ${s3Secret} -binary | base64`
curl -H "Host: ${bucket}.s3.amazonaws.com" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \ 
-H "Authorization: AWS ${s3Key}:${signature}" \ 
https://${bucket}.s3.amazonaws.com/${file}

Hope it helps.

Kannaiyan
  • 12,554
  • 3
  • 44
  • 83
  • 3
    curl: (6) Could not resolve host: ./curl-s3-with-keysAuth.sh: 17: ./curl-s3-with-keysAuth.sh: -H: not found I keep getting this error: though i verified my path is correct. – AhmFM Sep 12 '19 at 21:24