0

I am trying to set an AWS Lambda function in Python, which is triggered during an IOT Analytics treatment and sends two successive POST requests to an external API (if a condition is met). I am unable to import the "request" package directly, as it is too large for inline editing, so I have uploaded my Python script as well as the package into the zip file, then uploading it to AWS.

When running the lambda, I get errors on the requests package files, which I don't really understand. I am also unsure about how to return the API response, as I get a serialization error on my response. Here is my lambda code:

import json
import os
import time
import requests

def lambda_handler(event,context):

headers = {
        "authorization": "Basic XXXXXXXXX",
        "content_type": "application/json"
        } #API request header

url = "https://example.com/path" #API request URL

msgs = json.loads(json.dumps(event))
for msg in msgs:
    deviceID = msg["deviceID"]
    data = msg["data"]

    if (condition over received message):
        body1 = {
            "paramAA": paramAA,
            "paramB": [{
                    "paramBA": paramBA1,
                    "paramBB": paramBB
                    }]
            }
        response_1 = requests.post(url,headers=headers,data=body1) #First API request

        time.sleep(600)

        body2 = {
            "paramAA": paramAA,
            "paramB": [{
                "paramBA": paramBA2,
                "paramBB": paramBB,
                }]
            }
        response_2 = requests.post(url,headers=headers,data=body2) #Second API request
    else:
        pass
else:
    pass

return {
    'statusCode': 200,
    'url' : url,
    'response_1.code' : response_1.status_code,
    'response_1_msg' : response_1.text,
    'response_2.code' : response_2.status_code,
    'response_2_msg' : response_2.text
    }

Do you have any idea how to fix these errors?

If I change the return with "reponse_1 : json.dumps(response1)", I get these errors (whether it is with the zip package or the SDK):

{
"errorMessage": "Object of type Response is not JSON serializable",
"errorType": "TypeError",
"stackTrace": [
  "  File \"/var/task/lambda_function.py\", line 56, in lambda_handler\n    'response_1' : json.dumps(response_1),\n",
  "  File \"/var/lang/lib/python3.8/json/__init__.py\", line 231, in dumps\n    return _default_encoder.encode(obj)\n",
  "  File \"/var/lang/lib/python3.8/json/encoder.py\", line 199, in encode\n    chunks = self.iterencode(o, _one_shot=True)\n",
  "  File \"/var/lang/lib/python3.8/json/encoder.py\", line 257, in iterencode\n    return _iterencode(o, 0)\n",
  "  File \"/var/lang/lib/python3.8/json/encoder.py\", line 179, in default\n    raise TypeError(f'Object of type {o.__class__.__name__} '\n"
MrCurious
  • 1
  • 3
  • What are the errors about the request package? The best practice for Lambda development is to bundle all dependencies used by your Lambda function, including the AWS SDK. By doing this, your code uses the bundled version and is not affected when the version in the execution environment is upgraded. This is preferable to using the included version of the SDK, since this version can change, and in rare cases might affect compatibility with your code. https://aws.amazon.com/blogs/compute/upcoming-changes-to-the-python-sdk-in-aws-lambda/ – Jeremy Thompson Mar 27 '20 at 00:00
  • Thanks for the recommendation! I have the following error : { "errorMessage": "Unable to marshal response: Object of type Response is not JSON serializable", "errorType": "Runtime.MarshalError" } – MrCurious Mar 28 '20 at 08:45
  • Use `response1.json()`? – DisappointedByUnaccountableMod Mar 28 '20 at 08:52
  • Hi Barny, error remains the same: { "errorMessage": "Unable to marshal response: Object of type Response is not JSON serializable", "errorType": "Runtime.MarshalError" } – MrCurious Mar 28 '20 at 10:31
  • Is `"paramAA": paramAA` correct syntax, does the : operator concatenate it? Eg ""paramAA"" + ":" + paramAA? And what happens if the paramAA variable is a string, shouldn't it be enclosed it double quotes? The error is saying badly formatted JSON so just whip up a 3 line project to check how you're formatting the JSON and run that through a website for JSON validation – Jeremy Thompson Mar 28 '20 at 12:26
  • I haven't detailed it but they are already indeed string values with quotes. I have tried the API request on a REST client, and it worked out successfully with these parameters. However, in my Lambda, if I remove all the "reponse" items in the return, the lambda returns a 200 code, but there is no request sent to the API in its logs (not even an error one). I am sure that the condition over the received message was fulfilled, as I created a test variable that changed in the "if" loop, which is correctly returned. – MrCurious Mar 28 '20 at 20:20

2 Answers2

0

It appears that json library can convert to string only standard types, but I had to use pickle for more complex objects. I have modified my code accordingly, and ended up with a utf-8 encoding error :

  "errorMessage": "Unable to marshal response: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte"

Here is the script that I have now:

import pickle
import os
import time
import requests

def lambda_handler(event,context):

headers = {
        "authorization": "Basic XXXXXXXXX",
        "content_type": "application/json"
        } #API request header

url = "https://example.com/path" #API request URL

msgs = pickle.loads(pickle.dumps(event))
for msg in msgs:
    deviceID = msg["deviceID"]
    data = msg["data"]

    if (condition over received message):
        body1 = {
            "paramAA": paramAA,
            "paramB": [{
                    "paramBA": paramBA1,
                    "paramBB": paramBB
                    }]
            }
        response_1 = requests.post(url,headers=headers,data=pickle.dumps(body1)) #First API request
        exec_verif = "request 1 sent"

        time.sleep(600)

        body2 = {
            "paramAA": paramAA,
            "paramB": [{
                "paramBA": paramBA2,
                "paramBB": paramBB,
                }]
            }
        response_2 = requests.post(url,headers=headers,data=pickle.dumps(body2)) #Second API request
        exec_verif = "request 2 sent"
    else:
        response_1 = "fail1"
        response_2 = "fail1"
else:
    response_1 = "fail2"
    response_2 = "fail2"

return {
    'statusCode': 200,
    'exec_verif' : exec_verif,
    'response_1' : pickle.dumps(response_1),
    'response_2' : pickle.dumps(response_2)
    }

Something strange to me is that, if I don't include response_1 and response_2 in the return, the function is executed and sends back a 200 status, with 'exec_verif' being "request 2 sent", while there is no actual call to the external API. There is no trace of any call, even unsuccessfull, in the API logs.

MrCurious
  • 1
  • 3
  • Finally solved it thanks to this post! https://stackoverflow.com/questions/9733638/post-json-using-python-requests Just needed to change the "data" parameter into "json" in the request – MrCurious Apr 06 '20 at 18:30
0

Using base64 encoding worked for me. An example is given here; https://docs.aws.amazon.com/apigateway/latest/developerguide/lambda-proxy-binary-media.html

More details here; https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings-workflow.html

ML_Life
  • 1
  • 1
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. – Tyler2P Apr 07 '22 at 18:57
  • While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - [From Review](/review/late-answers/31481137) – Omar Tammam Apr 10 '22 at 22:26