troubles generating signature for alibaba cloud

Question

Reading the HTTP API docs. My requests fail though for bad signature. From error message I can see that my string to sign is correct but looks like I can't generate the correct HMAC-SHA1 (seriously why use SHA1 still??).

So I decided to try replicate the signature of the sample inside same document.

[47] pry(main)> to_sign = "GET&%2F&AccessKeyId%3Dtestid&Action%3DDescribeRegions&Format%3DXML&SignatureMethod%3DHMAC-SHA1&SignatureNonce%3D3ee8c1b8-83d3-44af-a94f-4e0ad82fd6cf&SignatureVersion%3D1.0&Timestamp%3D2016-02-23T12%253A46%253A24Z&Version%3D2014-05-26"

[48] pry(main)> Base64.encode64 OpenSSL::HMAC.digest("sha1", "testsecret", to_sign)
=> "MLAxpXej4jJ7TL0smgWpOgynR7s=\n"

[49] pry(main)> Base64.encode64 OpenSSL::HMAC.digest("sha1", "testsecret&", to_sign)
=> "VyBL52idtt+oImX0NZC+2ngk15Q=\n"

[50] pry(main)> Base64.encode64 OpenSSL::HMAC.hexdigest("sha1", "testsecret&", to_sign)
=> "NTcyMDRiZTc2ODlkYjZkZmE4MjI2NWY0MzU5MGJlZGE3ODI0ZDc5NA==\n"

[51] pry(main)> Base64.encode64 OpenSSL::HMAC.hexdigest("sha1", "testsecret", to_sign)
=> "MzBiMDMxYTU3N2EzZTIzMjdiNGNiZDJjOWEwNWE5M2EwY2E3NDdiYg==\n"

[52] pry(main)> OpenSSL::HMAC.hexdigest("sha1", "testsecret&", to_sign)
=> "57204be7689db6dfa82265f43590beda7824d794"

[53] pry(main)> OpenSSL::HMAC.hexdigest("sha1", "testsecret", to_sign)
=> "30b031a577a3e2327b4cbd2c9a05a93a0ca747bb"

As evident none of these matches the example signature of CT9X0VtwR86fNWSnsc6v8YGOjuE=. Any idea what is missing here?

Update: taking tcpdump from the Golang client tool I see that it does a POST request like:

POST /?AccessKeyId=**********&Action=DescribeRegions&Format=JSON&RegionId=cn-qingdao&Signature=aHZVpIMb0%2BFKdoWSIVaFJ7bd2LA%3D&SignatureMethod=HMAC-SHA1&SignatureNonce=c29a0e28964c470a8997aebca4848b57&SignatureType=&SignatureVersion=1.0&Timestamp=2018-07-16T19%3A46%3A33Z&Version=2014-05-26 HTTP/1.1

    Host: ecs.aliyuncs.com
    User-Agent: Aliyun-CLI-V3.0.3
    Content-Length: 0
    Content-Type: application/x-www-form-urlencoded
    x-sdk-client: golang/1.0.0
    x-sdk-core-version: 0.0.1
    x-sdk-invoke-type: common
    Accept-Encoding: gzip

When I take parameters from the above request and generate signature it does match. So I tried all tree: GET, POST with URL params and POST with params in body. Every time I am getting a signature error. If I redo the request with exact same params as the golang tool, I'm getting nonce already used error (as expected).

Answer 1

Finally got this working. The main issue in my case was that I have been double-percent-encoding the signature parameter thus it turned out invalid. What helped me most was running the aliyun cli utility and capturing traffic, then running a query with exactly the same parameters to compare the exact query string.

But let me list some key points for me:

1. once hmac-sha1 sig is generated, do not percent-encode it, just add it to the query with normal form www encoding
2. order of parameters in the HTTP query is not significant; order of parameters in the signing string is significant though
3. I find all the following types of requests to work: GET, POST with parameters in URL query, POST with parameters in request body form www encoded; I'm using GET per documentation but I see aliyun using POST vs query params and ordered params in the query
4. you must add & character to the end of the secret key when generating HMAC-SHA1
5. generate HMAC-SHA1 in binary form, then encode as Base64 (no hex values)
6. some parameters might be case insensitive, e.g. Format works both as json and JSON
7. I see aliyun, @wanghq and John using UUID 4 for SignatureNonce but I deferred to plain random (according to docs) because it seems to be only a replay attack protection. So cryptographically secure random number must unnecessary.
8. The special encoding rules for +, * and ~ seem to only apply to string for signing, not actually to encode data in such a way in the HTTP query.

I decided to not use @wanghq's wrapper as it didn't work for me as well disables certificate validation but maybe it's going to be fixed. Just I thought that queries are simple enough once signature is figured out and an additional layer of indirection is not worth it. +1 to his answer though as it was helpful to get my signature right.

Here's example ruby code to make a simple request:

require 'base64'
require 'cgi'
require 'openssl'
require 'time'
require 'rest-client'

# perform a request against Alibaba Cloud API
# @see https://www.alibabacloud.com/help/doc-detail/25489.htm
def request(action:, params: {})
  api_url = "https://ecs.aliyuncs.com/"

  # method = "POST"
  method = "GET"
  process_params!(http: method, action: action, params: params)
  RestClient::Request.new(method: method, url: api_url, headers: {params: params})
  # RestClient::Request.new(method: method, url: api_url, payload: params)
  # RestClient::Request.new(method: method, url: api_url, payload: params.map{|k,v| "#{k}=#{CGI.escape(v)}"}.join("&"))
end

# generates the required common params for a request and adds them to params
# @return undefined
# @see https://www.alibabacloud.com/help/doc-detail/25490.htm
def process_params!(http:, action:, params:)
  params.merge!({
    "Action" => action,
    "AccessKeyId" => config[:auth][:key_id],
    "Format" => "JSON",
    "Version" => "2014-05-26",
    "Timestamp" => Time.now.utc.iso8601
  })
  sign!(http: http, action: action, params: params)
end

# generate request signature and adds to params
# @return undefined
# @see https://www.alibabacloud.com/help/doc-detail/25492.htm
def sign!(http:, action:, params:)
  params.delete "Signature"
  params["SignatureMethod"] = "HMAC-SHA1"
  params["SignatureVersion"] = "1.0"
  params["SignatureNonce"] = "#{rand(1_000_000_000_000)}"
  # params["SignatureNonce"] = SecureRandom.uuid.gsub("-", "")

  canonicalized_query_string = params.sort.map { |key, value|
    "#{key}=#{percent_encode value}"
  }.join("&")

  string_to_sign = %{#{http}&#{percent_encode("/")}&#{percent_encode(canonicalized_query_string)}}

  params["Signature"] = hmac_sha1(string_to_sign)
end

# @param data [String]
# @return [String]
def hmac_sha1(data, secret: config[:auth][:key_secret])
  Base64.encode64(OpenSSL::HMAC.digest('sha1', "#{secret}&", data)).strip
end

# encode strings per Alibaba cloud rules for signing
# @return [String] encoded string
# @see https://www.alibabacloud.com/help/doc-detail/25492.htm
def percent_encode(str)
  CGI.escape(str).gsub(?+, "%20").gsub(?*, "%2A").gsub("%7E", ?~)
end

## example call
request(action: "DescribeRegions")

Code can be simplified a little but decided to keep it very close to documentation instructions.

P.S. not sure why John deleted his answer but leaving a link above to his web page for any python guys looking for example code

Answer 2

• Seems this aliyun ruby sdk (non official, just for reference) works. You may want to check how it's implemented.
• Check how its string_to_sign looks like. I did a run and seems it's slightly different than what you provided. The params are concatenated with & instead of %26. GET&%2F&AccessKeyId%3Dtestid&Action%3DDescribeRegions&Format%3DXML&SignatureMethod%3DHMAC-SHA1&SignatureNonce%3D3ee8c1b8-83d3-44af-a94f-4e0ad82fd6cf&SignatureVersion%3D1.0&Timestamp%3D2016-02-23T12%253A46%253A24Z&Version%3D2014-05-26

    require 'rubygems'
    require 'aliyun'

    $DEBUG = true

    options = {
      :access_key_id => "k",
      :access_key_secret => "s",
      :service => :ecs
    }

    service = Aliyun::Service.new options

    puts service.DescribeRegions({})

Answer 3

wanted to share a library I found (Python) that does everything for me w/o the need to sign the request myself. It can also help those who wants to just copy their functions and still construct the signature on their own

I'm using this:

from aliyunsdkcore.client import AcsClient
from aliyunsdkvpc.request.v20160428.DescribeEipAddressesRequest import DescribeEipAddressesRequest

client = AcsClient(access_key, secret_key, region)
request = DescribeEipAddressesRequest()
request.set_accept_format('json')

response = client.do_action_with_exception(request) # FYI returned as Bytes
print(response)

Each section in Alibaba Cloud has its own library (just like I used: aliyunsdkvpc for EIP addresses) And they are all listed here: https://develop.aliyun.com/tools/sdk?#/python