Random "SignatureDoesNotMatch" forbidden requests even when uploading same file via POST to Amazon S3

Question

I am uploading the same file multiple times "File.txt". Sometimes it successfully uploads to S3 and sometimes it raises:

SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. Check your key and signing method

I don't know what to test to discover the problem. Is it really a policy + signature problem as the exception says? So why sometimes can I upload?

Before upgrading to Rails 4 and updating some libs the same configurations always worked. But the updates in this case not even affect a simple POST form...

The POST params request sent with a failed upload were:

key: attachments/ad6d5c8c-f9ae-48ab-a9ff-c1b199a91d1d/File.txt

acl: public-read

policy: XXX==

signature: YYY=

Content-Type: binary/octet-stream

AWSAccessKeyId: -- Hidden --

...And with a successful one:

key: attachments/368f5497-6e11-4f07-b379-80020e902013/File.txt

acl: public-read

policy: XXX==

signature: ZZZ=

Content-Type: binary/octet-stream

AWSAccessKeyId: -- Hidden --

It seems right just like before. The only attribute that changed was the signature because of the random guid generation in the path...

Here are some other methods for generation policies:

def fields
    {
      key: key,
      acl: acl,
      policy: policy,
      signature: signature,
      "Content-Type" => content_type || "binary/octet-stream",
      "AWSAccessKeyId" => S3_CONFIG["access_key_id"]
    }
  end

  def policy
    Base64.encode64(policy_data.to_json).gsub("\n", "")
  end

  def policy_data
    {
      expiration: expiration,
      conditions: [
        {bucket: S3_CONFIG["bucket"]},
        ["starts-with", "$key", store_dir],
        {acl: acl},
        ["starts-with", "$Content-Type", ''],
        ["content-length-range", min_file_size, max_file_size]
      ]
    }
  end

  def signature
    Base64.encode64(
      OpenSSL::HMAC.digest(
        OpenSSL::Digest::Digest.new('sha1'),
        S3_CONFIG["secret_access_key"], policy
      )
    ).gsub("\n", "")
  end

Any help I appreciate Thanks

Shouldn’t [this](http://stackoverflow.com/questions/16782198/cannot-load-railtie-after-upgrade-to-rails-4-per-ruby-railstutorial-org) help you? — Aleksei Matiushkin, Jan 28 '14 at 03:01
Thanks for the answer, but it is an error outside rails. It comes as response callback from the amazon S3 upload — Luccas, Jan 28 '14 at 03:07
Oooups; sorry, for an unknown reason I copy-pasted wrong reference. Here you do: http://stackoverflow.com/questions/17397924/amazon-s3-strange-error-sometimes-signaturedoesnotmatch-sometimes-it-does — Aleksei Matiushkin, Jan 28 '14 at 03:11
no problem. I saw this question too before, but checked my signature and both doesn't have "+". — Luccas, Jan 28 '14 at 03:14
Another suspicious thing is `.gsub("\n", "")` in `policy`. Is this code being run on Unix? Maybe gsubbing `"\r\n"` would be better? — Aleksei Matiushkin, Jan 28 '14 at 03:19
Yes, I'm running from a Mac Os. Tried with "\r\n" but this adds "\n" to the end of the policy... — Luccas, Jan 28 '14 at 03:36
Still with my code above, I found a case of failed upload that my signature now presented a "+".. What should I do? Escaping the entire signature with CGI.escape(...) does not work too. — Luccas, Jan 28 '14 at 03:40

score 0 · Answer 1 · edited May 23 '17 at 12:28

Let me try to sum up the things. This would be a draft of solution only, since I don’t have the env here to test it.

• According to this, one should encode the signature with url encode:

require 'open-uri'
signature = URI::encode(signature)

• Gsubbing "\n" in policy/signature looks a bit suspicious. I would try to either not touch it at all, or to remove all possible LF/CR on all the operating system environments with .gsub("\n|\r")

I would start trying with encoded not gsubbed policy/signature and then continue with gsubbed versions.

Hope it helps.

score 0 · Answer 2 · answered May 14 '14 at 15:20

I had the same issue but I figured it out by printing the decoding the 'policy' in the fields (form) and in signature.

The answer is that the policy method is called twice (in fields and in signature), each time the policy is generated with different expiration value. That's why it's random and doesn't happen every time.

The solution is easy - keep policy in a variable for later access:

  def policy
    @policy ||= Base64.encode64(policy_data.to_json).gsub("\n", "")
  end

A fixed expiration value should work as well.

Random "SignatureDoesNotMatch" forbidden requests even when uploading same file via POST to Amazon S3

2 Answers2