0

I am trying to PUT files from EC2 to S3 using bash/curl and instance profile. I am using the following code:

instance_profile=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/`
aws_access_key_id=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep AccessKeyId | cut -d':' -f2 | sed 's/[^0-9A-Z]*//g'`
aws_secret_access_key=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep SecretAccessKey | cut -d':' -f2 | sed 's/[^0-9A-Za-z/+=]*//g'
token=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | sed -n '/Token/{p;}' | cut -f4 -d'"'
file="test_file.txt"
bucket="MM-test-s3-bucket"
filepath="/${bucket}/${path}/${file}"
contentType="application/x-compressed-tar"
dateValue=`date -R`
signature_string="PUT\n\n${contentType}\n${dateValue}\n${filepath}"
signature_hash=`echo -en ${signature_string} | openssl sha1 -hmac ${aws_secret_access_key} -binary | base64`

curl -X PUT -T "${file}" -H "Host: ${bucket}.s3.amazonaws.com" -H "Date: ${dateValue}" -H "Content-Type: ${contentType}" -H "Authorization: AWS ${aws_access_key_id}:${signature_hash}" https://${bucket}.s3.amazonaws.com/${file}

I am getting an error "InvalidAccessKeyIdThe AWS Access Key Id you provided does not exist in our records."

Muneeb
  • 99
  • 8
  • What's the output of `echo $aws_access_key_id`? – jellycsc Jul 12 '22 at 12:22
  • It shows the access key starting with ASIA******** – Muneeb Jul 12 '22 at 12:32
  • 1
    Since the access key starts with ASIA, it's a temporary credentials. You also need to include the security token in the request. – jellycsc Jul 12 '22 at 12:48
  • Thanks! That issue is gone. Now getting the error "The request signature we calculated does not match the signature you provided. Check your key and signing method." – Muneeb Jul 12 '22 at 13:39
  • This is probably related to this: https://stackoverflow.com/questions/30518899/amazon-s3-how-to-fix-the-request-signature-we-calculated-does-not-match-the-s – X-Men Jul 13 '22 at 03:25
  • That issue is also gone. Another I am getting now: "Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4" – Muneeb Jul 13 '22 at 13:26

1 Answers1

1

As mentioned in the questions' comment, we need to use token when using temporary access keys and secret key.

Also, we need to add this token in our stringToSign and headers in curl request.

stringToSign="PUT\n\n${contentType}\n${dateValue}\n${CanonicalizedAmzHeaders}\n${filepath}".

Where this CanonicalizedAmzHeaders is - CanonicalizedAmzHeaders="x-amz-security-token:${token}"

and in header in curl request in below format- -H "x-amz-security-token: ${token}"

else you will face error "The request signature we calculated does not match the signature you provided"

Akshay Verma
  • 21
  • 2