AWS authentication V4 signature failure; where am I going wrong in generating the signature?

Question

I am generating a form using the ruby code below (passing the CSV file with credentials downloads from the AWS console as the argument). If I submit a file using this form, I get The request signature we calculated does not match the signature you provided. Check your key and signing method.. I have copied the signing code from http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-ruby. I've looked at Amazon MWS - request signature calculated does not match the signature provided and s3 "signature doesn't match" client side post jquery-file-upload, but these don't seem to apply in my case. Where am I going wrong?

#!/usr/bin/env ruby

require 'nokogiri'
require 'csv'
require 'ostruct'
require 'base64'
require 'json'
require 'openssl'

header = nil
data = nil
CSV.foreach(ARGV[0]) do |row|
  if header.nil?
    header = row.collect{|c| c.strip.gsub(/\s/, '') }
  else
    data = row
  end
end
creds = OpenStruct.new(Hash[*(header.zip(data).flatten)])

bucket = 'zotplus'
region = 'eu-central-1'
service = 's3'
dateStamp = Time.now.strftime('%Y%m%d')

policy = {
  'expiration' => '2029-01-01T00:00:00Z',
  'conditions' => [
    {'bucket' => bucket},
    ['starts-with', '$key', 'uploads/'],
    {'acl' => 'private'},
    {'success_action_redirect' => 'http://zotplus.github.io/submitted.html'},
    ['starts-with', '$Content-Type', 'multipart/form-data'],
    ['content-length-range', 0, 1048576],
    {'x-amz-date' => "#{dateStamp}T000000Z"},
    {'x-amz-credential' => "#{creds.AccessKeyId}/#{dateStamp}/#{region}/#{service}/aws4_request"}
  ]
}

form = {}
%w{acl success_action_redirect bucket x-amz-date x-amz-credential}.each{|eq|
  form[eq] = policy['conditions'].detect{|c| c.is_a?(Hash) && c[eq] }[eq]
}
form['key'] = policy['conditions'].detect{|c| c.is_a?(Array) && c[0,2] = ['starts-with', '$key']}[2] + '${filename}'

policy_string = Base64.encode64(policy.to_json).gsub("\n","")

kDate    = OpenSSL::HMAC.digest('sha256', "AWS4" + creds.SecretAccessKey, dateStamp)
kRegion  = OpenSSL::HMAC.digest('sha256', kDate, region)
kService = OpenSSL::HMAC.digest('sha256', kRegion, service)
kSigning = OpenSSL::HMAC.digest('sha256', kService, 'aws4_request')

signature = Base64.encode64(OpenSSL::HMAC.digest('sha256', kSigning, policy_string)).gsub("\n","")

form['policy'] = policy_string
form['x-amz-signature'] = signature
form['x-amz-algorithm'] = 'AWS4-HMAC-SHA256'

builder = Nokogiri::HTML::Builder.new do |doc|
  doc.html {
    doc.head {
      doc.title {
        doc.text 'submit file'
      }
      doc.meta('http-equiv' => "Content-Type", content: "text/html; charset=UTF-8")
    }

    doc.body {
      doc.form(action: "http://zotplus-964ec2b7-379e-49a4-9c8a-edcb20db343f.s3.amazonaws.com/", method: "post", enctype: "multipart/form-data") {
        form.each_pair{|k, v|
          doc.input(type: "hidden", name: k, value: v)
        }

        doc.text 'File: '
        doc.input(type: "file", name: "file")
        doc.input(type: "submit", name: "submit", value: "Upload to Amazon S3")
      }
    }
  }
end
puts builder.to_html

Grab a copy of the [S3 Signature V4 Test suite](http://docs.aws.amazon.com/general/latest/gr/signature-v4-test-suite.html) to see how you are doing. Importantly, the test suite contains the results you should be obtaining at the intermediate steps (e.g., canonical request and string to sign), so it's relatively straightforward to find the error. If not, you might be able to at least narrow the focus of your question to the specific part where the first mismatch occurs. — Michael - sqlbot, Jun 25 '15 at 22:03

score 1 · Answer 1 · answered Jun 26 '15 at 08:38

OK, it turns out the policy signature should not be base64 encoded but simply output as a hex digest. For those struggling with the same problem, this is what worked in the end:

#!/usr/bin/env ruby

require 'json'
require 'base64'
require 'openssl'
require 'csv'
require 'ostruct'
require 'nokogiri'

algorithm = 'AWS4-HMAC-SHA256'
service = 's3'
requestType = 'aws4_request'
successStatus = '201'
bucket = 'zotplus-964ec2b7-379e-49a4-9c8a-edcb20db343f'
region = 'eu-central-1'
acl = 'private'

header = nil
data = nil
CSV.foreach(ENV['ZOTPLUSAWSCREDENTIALS']) do |row|
  if header.nil?
    header = row.collect{|c| c.strip.gsub(/\s/, '') }
  else
    data = row
  end
end
creds = OpenStruct.new(Hash[*(header.zip(data).flatten)])

date = Time.now.strftime('%Y%m%dT%H%M%SZ')
shortDate = date.sub(/T.*/, '')
credentials = [ creds.AccessKeyId, shortDate, region, service, requestType ].join('/')

policy = Base64.encode64({
  'expiration' => (Time.now + (60*60*24*365*30)).strftime('%Y-%m-%dT%H:%M:%SZ'), # 30 years from now
  'conditions' => [
    {'bucket' => bucket},
    {'acl' => acl},
    ['starts-with', '$key', ''],
    ['starts-with', '$Content-Type', ''],
    {'success_action_status' => successStatus},
    {'x-amz-credential' => credentials},
    {'x-amz-algorithm' => algorithm},
    {'x-amz-date' => date},
    ['content-length-range', 0, 1048576],
  ]
}.to_json).gsub("\n","")

signingKey = ['AWS4' + creds.SecretAccessKey, shortDate, region, service, requestType].inject{|key, data| OpenSSL::HMAC.digest('sha256', key, data) } 

form = OpenStruct.new({
  action: "http://#{bucket}.#{service}-#{region}.amazonaws.com",
  fields: [ # order matters!
    {key: '${filename}'},
    {'Content-Type': 'text/plain'},
    {acl: acl},
    {success_action_status: successStatus},
    {policy: policy},
    {'x-amz-algorithm': algorithm},
    {'x-amz-credential': credentials},
    {'x-amz-date': date},
    {'x-amz-signature': OpenSSL::HMAC.hexdigest('sha256', signingKey, policy)}
  ]
})

################################################

builder = Nokogiri::HTML::Builder.new do |doc|
  doc.html {
    doc.head {
      doc.meta(charset: 'utf-8')
      doc.title { doc.text 'Upload' }
    }
    doc.body {
      doc.form(action: form.action, method: 'POST', enctype: "multipart/form-data") {
        form.fields.each{|field|
          field.each_pair{|name, value|
            doc.input(type: 'hidden', name: name, value: value)
          }
        }
        doc.input(type: 'file', name: 'file')
        doc.input(type: 'submit', value: 'Save')
      }
    }
  }
end
puts builder.to_html

score 0 · Answer 2 · answered Jul 01 '16 at 11:31

Yea it works :D I tried it this way

def index
access_key = 'YOUR_ACCESS_KEY'
secret_key = 'YOUR_SECRET_KEY'
time = Time.now.utc
date_stamp = time.strftime("%Y%m%d")
region_name = 'ap-south-1'
key_date    = hmac_digest('sha256', "AWS4" + secret_key, date_stamp)
key_region  = hmac_digest('sha256', key_date, region_name)
key_service = hmac_digest('sha256', key_region, 's3')
key_signing = hmac_digest('sha256', key_service, "aws4_request")
algorithm = 'AWS4-HMAC-SHA256'
amzdate = time.strftime('%Y%m%dT%H%M%SZ')
credential_scope = access_key + '/' + date_stamp + '/ap-south-1/s3/aws4_request'
policy = generate_policy(credential_scope, algorithm, amzdate)
signature = OpenSSL::HMAC.hexdigest('sha256', key_signing, policy)
render json: { policy: policy, signature: signature, key: access_key, date: amzdate, credentials: credential_scope, algorithm: algorithm }
end
def generate_policy(credential_scope, algorithm, amzdate)
Base64.encode64({
  'expiration' => (Time.now + (60 * 60 * 24 * 365 * 30)).strftime('%Y-%m-%dT%H:%M:%SZ'), # 30 years from now
  'conditions' => [
    { 'bucket' => 'adcreation-m' },
    { 'acl' => 'public-read' },
    ['starts-with', '$key', ''],
    ['starts-with', '$Content-Type', ''],
    { 'success_action_status' => '201' },
    { 'x-amz-credential' => credential_scope },
    { 'x-amz-algorithm' => algorithm },
    { 'x-amz-date' => amzdate },
    ['content-length-range', 0, 256000000]
  ]
}.to_json).delete("\n")
end

def hmac_digest(digest, key, data)
 OpenSSL::HMAC.digest(digest, key, data)
end

AWS authentication V4 signature failure; where am I going wrong in generating the signature?

2 Answers2