Error: InvalidSignature when trying to connect to SP-API amazon

Question

I've trying to connect to amazon api for a week now. I've got stuck in this error and after readig the doc several times I can't realize which is the problem.

Here is my code:

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0

"""
Important

The AWS SDKs sign API requests for you using the access key that you specify when you
configure the SDK. When you use an SDK, you don’t need to learn how to sign API requests.
We recommend that you use the AWS SDKs to send API requests, instead of writing your own code.

The following example is a reference to help you get started if you have a need to write
your own code to send and sign requests. The example is for reference only and is not
maintained as functional code.
"""

# AWS Version 4 signing example

# EC2 API (DescribeRegions)

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac 
import requests # pip install requests

# ************* REQUEST VALUES *************
method = 'GET'
service = 'execute-api'
host = 'sellingpartnerapi-na.amazon.com'
region = 'us-east-1'
endpoint = 'https://sellingpartnerapi-na.amazon.com'
request_parameters = 'Action=ListOrders&MarketplaceId=ATVPDKIKX0DER&Version=0'

#service = 'ec2'
#host = 'ec2.amazonaws.com'
#region = 'us-east-1'
#endpoint = 'https://ec2.amazonaws.com'
#request_parameters = 'Action=DescribeRegions&Version=2013-10-15'



# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = 'AKIEXAMPLE'
secret_key = 'SECRETEXAMPLE'
if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = '/orders/v0/orders' 

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 4: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n'

# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;x-amz-date'

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()

# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash


# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()


# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {'x-amz-date':amzdate, 'Authorization':authorization_header}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
r = requests.get(request_url, headers=headers)

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)

My application is originally built in Java, but since I've got the same error in the Python code sample from amazon, I'm tring to make it work first in Python.

It's also interesting that if I uncomment the code:

#service = 'ec2'
#host = 'ec2.amazonaws.com'
#region = 'us-east-1'
#endpoint = 'https://ec2.amazonaws.com'
#request_parameters = 'Action=DescribeRegions&Version=2013-10-15'

It works, but if I use my own endpoints it doesn't. I've checked everything and tried a lot of things, any idea of why this is happening? Thanks in advance for your time.

The full error msg
{
  "errors": [
    {
      "message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

     "code": "InvalidSignature"
    }
  ]
}

Answer 1

This is another solution without boto3 and only using requests

import hashlib
import hmac
import logging
from collections import OrderedDict
from urllib.parse import urlencode
import defusedxml.ElementTree as ET
from sdc_etl_libs.api_helpers.API import API
import sys, datetime, hashlib, hmac 
import requests
import json
from bs4 import BeautifulSoup
def get_session_token_from_xml(content):
    soup = BeautifulSoup(content, "xml")
    return soup.find('SessionToken').text, soup.find('AccessKeyId').text, soup.find('SecretAccessKey').text

def set_params(action_):

    logging.info(f"Setting params according to action {action_}")
    params = dict()
    if action_ == 'AssumeRole':
        params['Version'] = '2011-06-15'
        params['Action'] = action_
        params['RoleSessionName'] = <<ROLE NAME>>
        params['RoleArn'] = <<ROLE ARN>>
        params['DurationSeconds']='3600'
    elif action_ == 'orders':
        params['MarketplaceIds'] = <<MARKET PLACE>>
        params['LastUpdatedAfter'] = '2022-11-27T14:00:00Z'
        params['LastUpdatedBefore'] = '2022-11-27T16:00:00Z'
    else:
        raise Exception("Action is not implemented.")
    return params
def _get_access_token(lwa_app_id, lwa_client_secret, refresh_token):
    url = "https://api.amazon.com/auth/O2/token"

    payload=f'client_id={lwa_app_id}&client_secret={lwa_client_secret}&refresh_token={refresh_token}&grant_type=refresh_token'
    headers = {
    'Host': 'api.amazon.com',
    'Content-Type': 'application/x-www-form-urlencoded',
    }

    response = requests.request("POST", url, headers=headers, data=payload)

    return response
def format_params_to_create_signature(params_to_format_):
    """
    URL encodes the parameter name and values
    https://docs.developer.amazonservices.com/en_US/dev_guide/DG_QueryString.html
    :param params_to_format_: dict. Parameters that should be ordered in natural byte order
    and url encoded.
    :return: str.
    """
    logging.info("Format params.")
    params_in_order = OrderedDict(sorted(params_to_format_.items()))
    params_formatted = urlencode(params_in_order, doseq=True)
    return params_formatted

def sign(key, msg):
    
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

def _get_signature_request(action, access_key, secret_key, service, host, region, endpoint, 
method: str = 'GET', access_token: str = None, security_token: str = None):
    
    # ************* REQUEST VALUES *************
    params = set_params(action)
    fparams = format_params_to_create_signature(params)
    request_parameters = fparams

    # Read AWS access key from env. variables or configuration file. Best practice is NOT
    # to embed credentials in code.
    if access_key is None or secret_key is None:
        raise Exception("Access key or secret key are not implemented.")

    # Create a date for headers and the credential string
    t = datetime.datetime.utcnow()
    amzdate = t.strftime('%Y%m%dT%H%M%SZ')
    datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope
    # ************* TASK 1: CREATE A CANONICAL REQUEST *************
    # http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

    # Step 1 is to define the verb (GET, POST, etc.)--already done.

    # Step 2: Create canonical URI--the part of the URI from domain to query 
    # string (use '/' if no path)
    if action == 'AssumeRole':
        canonical_uri = '/' 
    else:
        canonical_uri = '/orders/v0/orders' 

    # Step 3: Create the canonical query string. In this example (a GET request),
    # request parameters are in the query string. Query string values must
    # be URL-encoded (space=%20). The parameters must be sorted by name.
    # For this example, the query string is pre-formatted in the request_parameters variable.
    canonical_querystring = request_parameters

    # Step 4: Create the canonical headers and signed headers. Header names
    # must be trimmed and lowercase, and sorted in code point order from
    # low to high. Note that there is a trailing \n.

    # Step 5: Create the list of signed headers. This lists the headers
    # in the canonical_headers list, delimited with ";" and in alpha order.
    # Note: The request can include any headers; canonical_headers and
    # signed_headers lists those that you want to be included in the 
    # hash of the request. "Host" and "x-amz-date" are always required.
    if action == 'AssumeRole':
        canonical_headers = 'host:' + host + '\n' + 'x-amz-date:' + amzdate + '\n'

        signed_headers = 'host;x-amz-date'
    else:
        canonical_headers = 'host:' + host + '\n' + 'x-amz-access-token:' + \
        access_token + '\n' + 'x-amz-date:' + amzdate + '\n' + 'x-amz-security-token:' + \
            security_token + '\n'

        signed_headers = 'host;x-amz-access-token;x-amz-date;x-amz-security-token'
   

    # Step 6: Create payload hash (hash of the request body content). For GET
    # requests, the payload is an empty string ("").
    payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()

    # Step 7: Combine elements to create canonical request
    canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + \
        signed_headers + '\n' + payload_hash

    # ************* TASK 2: CREATE THE STRING TO SIGN*************
    # Match the algorithm to the hashing algorithm you use, either SHA-1 or
    # SHA-256 (recommended)
    algorithm = 'AWS4-HMAC-SHA256'
    credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
    string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' + \
        hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()

    # ************* TASK 3: CALCULATE THE SIGNATURE *************
    # Create the signing key using the function defined above.
    signing_key = getSignatureKey(secret_key, datestamp, region, service)
    # Sign the string_to_sign using the signing_key
    signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()
    
    # ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
    # The signing information can be either in a query string value or in 
    # a header named Authorization. This code shows how to use a header.
    # Create authorization header and add to request headers
    authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  \
        'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature
    # The request can include any headers, but MUST include "host", "x-amz-date", 
    # and (for this scenario) "Authorization". "host" and "x-amz-date" must
    # be included in the canonical_headers and signed_headers, as noted
    # earlier. Order here is not significant.
    # Python note: The 'host' header is added automatically by the Python 'requests' library.
    if action == 'AssumeRole':
        headers = {'x-amz-date':amzdate, 'Authorization':authorization_header}
    else:
        headers = {
            'authorization': authorization_header,
            'host': host,
            'x-amz-access-token': access_token,
            'x-amz-date': amzdate, 
            'x-amz-security-token': security_token
        }

    # ************* SEND THE REQUEST *************
    request_url = endpoint + '?' + canonical_querystring
    logging.info(f"BEGIN REQUEST++++++++++++++++++++++++++++++++++++'")
    logging.info(f"Request URL = {request_url}")
    r = requests.get(request_url, headers=headers)

    logging.info('\nRESPONSE++++++++++++++++++++++++++++++++++++')
    logging.info('Response code: %d\n' % r.status_code)

    return r

the way to run in is the following

service = 'sts'
host = 'sts.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://sts.amazonaws.com'
response = _get_signature_session('AssumeRole', access_key, secret_key, service, host, region, endpoint)
access_token = json.loads(_get_access_token(lwa_app_id, lwa_client_secret, refresh_token).content)['access_token']
tmp_session_token_, tmp_access_key, tmp_secret_access_key = get_session_token_from_xml(response.content.decode('utf-8'))

after that you will have the temporal session token, the temporal access key ath the temporal secret key. finally to get the all orders is in the following code

service = 'execute-api'
host = 'sellingpartnerapi-na.amazon.com'
region = 'us-east-1'
endpoint = 'https://sellingpartnerapi-na.amazon.com/orders/v0/orders'
response = _get_signature_session('orders', tmp_access_key, tmp_secret_access_key, service, host, region, endpoint,
 access_token = access_token, security_token = tmp_session_token_)

Answer 2

After some research and testing, I modified the python app, and it works!

Before reading the code below, READ THIS. You must execute pip install boto3 to make it work. Here are the docs: https://pypi.org/project/boto3/

I'm putting the credentials in a raw dict instead of following the boto3 docs structure because it was just for testing. If you want to test it with the code, just replace the credentials dict values.

Notice that it is working with the sandbox environment and getOrders endpoint, and you must specify your own RoleSessionName.

Here is the code:

# AWS Version 4 signing example

# EC2 API (DescribeRegions)

# See: http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html
# This version makes a GET request and passes the signature
# in the Authorization header.
import sys, os, base64, datetime, hashlib, hmac 
import requests # pip install requests
import boto3

credentials = {
    
    'lwa_refresh_token': 'whatever',
    'lwa_client_secret': 'whatever',
    'lwa_client_id': 'whatever',
    'aws_secret_access_key': 'whatever',
    'aws_access_key': 'whatever',
    'role_arn': 'whatever:role/whatever'
}


# get Access Token and assign to 'x-amz-access-token'
response = requests.post('https://api.amazon.com/auth/o2/token',
    headers={'Content-Type': 'application/x-www-form-urlencoded'},
    data={
        'grant_type': 'refresh_token',
        'refresh_token': credentials['lwa_refresh_token'],
        'client_id': credentials['lwa_client_id'],
        'client_secret': credentials['lwa_client_secret']
    }
)
credentials['x-amz-access-token'] = response.json()['access_token']

# get AWS STS Session Token and assign to 'x-amz-security-token'
sts_client = boto3.client(
    'sts',
    aws_access_key_id=credentials['aws_access_key'],
    aws_secret_access_key=credentials['aws_secret_access_key']
)

assumed_role_object=sts_client.assume_role(
    RoleArn=credentials['role_arn'],
    RoleSessionName="whatever role sesion name you got"
)
credentials['x-amz-security-token'] = assumed_role_object['Credentials']['SessionToken']
credentials['aws_access_key'] = assumed_role_object['Credentials']['AccessKeyId']
credentials['aws_secret_access_key'] = assumed_role_object['Credentials']['SecretAccessKey']

# ************* REQUEST VALUES *************
method = 'GET'
service = 'execute-api'
host = 'sandbox.sellingpartnerapi-na.amazon.com'
region = 'us-east-1'
endpoint = 'https://sandbox.sellingpartnerapi-na.amazon.com/orders/v0/orders'
request_parameters = 'CreatedAfter=TEST_CASE_200&MarketplaceIds=ATVPDKIKX0DER'

# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
    return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()

def getSignatureKey(key, dateStamp, regionName, serviceName):
    kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
    kRegion = sign(kDate, regionName)
    kService = sign(kRegion, serviceName)
    kSigning = sign(kService, 'aws4_request')
    return kSigning

# Read AWS access key from env. variables or configuration file. Best practice is NOT
# to embed credentials in code.
access_key = credentials['aws_access_key']
# No deberia de ser security-token, si no secret_access_key?¿
secret_key = credentials['aws_secret_access_key']
if access_key is None or secret_key is None:
    print('No access key is available.')
    sys.exit()

# Create a date for headers and the credential string
t = datetime.datetime.utcnow()
amzdate = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d') # Date w/o time, used in credential scope


# ************* TASK 1: CREATE A CANONICAL REQUEST *************
# http://docs.aws.amazon.com/general/latest/gr/sigv4-create-canonical-request.html

# Step 1 is to define the verb (GET, POST, etc.)--already done.

# Step 2: Create canonical URI--the part of the URI from domain to query 
# string (use '/' if no path)
canonical_uri = '/orders/v0/orders' 

# Step 3: Create the canonical query string. In this example (a GET request),
# request parameters are in the query string. Query string values must
# be URL-encoded (space=%20). The parameters must be sorted by name.
# For this example, the query string is pre-formatted in the request_parameters variable.
canonical_querystring = request_parameters

# Step 4: Create the canonical headers and signed headers. Header names
# must be trimmed and lowercase, and sorted in code point order from
# low to high. Note that there is a trailing \n.
canonical_headers = 'host:' + host + '\n' + 'user-agent:' + 'Ladder data ingestion' + '\n' + 'x-amz-access-token:' + credentials['x-amz-access-token'] + '\n' + 'x-amz-date:' + amzdate + '\n' + 'x-amz-security-token:' + credentials['x-amz-security-token'] + '\n'
    
# Step 5: Create the list of signed headers. This lists the headers
# in the canonical_headers list, delimited with ";" and in alpha order.
# Note: The request can include any headers; canonical_headers and
# signed_headers lists those that you want to be included in the 
# hash of the request. "Host" and "x-amz-date" are always required.
signed_headers = 'host;user-agent;x-amz-access-token;x-amz-date;x-amz-security-token'

# Step 6: Create payload hash (hash of the request body content). For GET
# requests, the payload is an empty string ("").
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()

# Step 7: Combine elements to create canonical request
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
print("My Canonical String:")
print(canonical_request+'\n')

# ************* TASK 2: CREATE THE STRING TO SIGN*************
# Match the algorithm to the hashing algorithm you use, either SHA-1 or
# SHA-256 (recommended)
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
string_to_sign = algorithm + '\n' +  amzdate + '\n' +  credential_scope + '\n' +  hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
print("My String to Sign")
print(string_to_sign+'\n')

# ************* TASK 3: CALCULATE THE SIGNATURE *************
# Create the signing key using the function defined above.
signing_key = getSignatureKey(secret_key, datestamp, region, service)

# Sign the string_to_sign using the signing_key
signature = hmac.new(signing_key, (string_to_sign).encode('utf-8'), hashlib.sha256).hexdigest()


# ************* TASK 4: ADD SIGNING INFORMATION TO THE REQUEST *************
# The signing information can be either in a query string value or in 
# a header named Authorization. This code shows how to use a header.
# Create authorization header and add to request headers
authorization_header = algorithm + ' ' + 'Credential=' + access_key + '/' + credential_scope + ', ' +  'SignedHeaders=' + signed_headers + ', ' + 'Signature=' + signature

# The request can include any headers, but MUST include "host", "x-amz-date", 
# and (for this scenario) "Authorization". "host" and "x-amz-date" must
# be included in the canonical_headers and signed_headers, as noted
# earlier. Order here is not significant.
# Python note: The 'host' header is added automatically by the Python 'requests' library.
headers = {
    'authorization': authorization_header,
    'host': host,
    'user-agent': 'Ladder data ingestion',
    'x-amz-access-token': credentials['x-amz-access-token'],
    'x-amz-date': amzdate, 
    'x-amz-security-token': credentials['x-amz-security-token']
}


# ************* SEND THE REQUEST *************
request_url = endpoint + '?' + canonical_querystring

print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
r = requests.get(request_url, headers=headers)

print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)

If you replace your credentials in the code, and it doesn't work, you may need to regenerate them. You can leave a comment here or open a new question and link it in a comment, so I can check it.

I've also made a request to getOrder() endpoint, let me know if you have any problem pointing to sandbox.