Can you have an async handler in Lambda Python 3.6?

Question

I've made Lambda functions before but not in Python. I know in Javascript Lambda supports the handler function being asynchronous, but I get an error if I try it in Python.

Here is the code I am trying to test:

async def handler(event, context):
    print(str(event))
    return { 
        'message' : 'OK'
    }

And this is the error I get:

An error occurred during JSON serialization of response: <coroutine object handler at 0x7f63a2d20308> is not JSON serializable
Traceback (most recent call last):
  File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/runtime/awslambda/bootstrap.py", line 149, in decimal_serializer
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <coroutine object handler at 0x7f63a2d20308> is not JSON serializable

/var/runtime/awslambda/bootstrap.py:312: RuntimeWarning: coroutine 'handler' was never awaited
  errortype, result, fatal = report_fault(invokeid, e)

---

EDIT 2021:

Since this question seems to be gaining traction, I assume people are coming here trying to figure out how to get async to work with AWS Lambda as I was. The bad news is that even now more than a year later, there still isn't any support by AWS to have an asynchronous handler in a Python-based Lambda function. (I have no idea why, as NodeJS-based Lambda functions can handle it perfectly fine.)

The good news is that since Python 3.7, there is a simple workaround in the form of asyncio.run:

import asyncio

def lambda_handler(event, context):
    # Use asyncio.run to synchronously "await" an async function
    result = asyncio.run(async_handler(event, context))
    
    return {
        'statusCode': 200,
        'body': result
    }

async def async_handler(event, context):
    # Put your asynchronous code here
    await asyncio.sleep(1)
    
    return 'Success'

Note: The selected answer says that using asyncio.run is not the proper way of starting an asynchronous task in Lambda. In general, they are correct because if some other resource in your Lambda code creates an event loop (a database/HTTP client, etc.), it's wasteful to create another loop and it's better to operate on the existing loop using asyncio.get_event_loop.

However, if an event loop does not yet exist when your code begins running, asyncio.run becomes the only (simple) course of action.

Answer 1

Not at all. Async Python handlers are not supported by AWS Lambda.

If you need to use async/await functionality in your AWS Lambda, you have to define an async function in your code (either in Lambda files or a Lambda Layer) and call asyncio.get_event_loop().run_until_complete(your_async_handler()) inside your regular sync Lambda handler:

import asyncio
import aioboto3

# To reduce execution time for subsequent invocations,
#   open a reusable resource in a global scope
dynamodb = aioboto3.Session().resource('dynamodb')

async def async_handler(event, context):
    # Put your asynchronous code here
    table = await dynamodb.Table('test')
    await table.put_item(
        Item={'pk': 'test1', 'col1': 'some_data'},
    )
    return {'statusCode': 200, 'body': '{"ok": true}'}

# Point to this function as a handler in the Lambda configuration
def lambda_handler(event, context):
    loop = asyncio.get_event_loop()
    # DynamoDB resource defined above is attached to this loop:
    #   if you use asyncio.run instead
    #   you will encounter "Event loop closed" exception
    return loop.run_until_complete(async_handler(event, context))

Please note that asyncio.run (introduced in Python 3.7) is not a proper way to call an async handler in AWS Lambda execution environment since Lambda tries to reuse the execution context for subsequent invocations. The problem here is that asyncio.run creates a new EventLoop and closes the previous one. If you have opened any resources or created coroutines attached to the closed EventLoop from previous Lambda invocation you will get «Event loop closed» error. asyncio.get_event_loop().run_until_complete allows you to reuse the same loop. See related StackOverflow question.

AWS Lambda documentation misleads its readers a little by introducing synchronous and asynchronous invocations. Do not mix it up with sync/async Python functions. Synchronous refers to invoking AWS Lambda with further waiting for the result (blocking operation). The function is called immediately and you get the response as soon as possible. Whereas using an asynchronous invocation you ask Lambda to schedule the function execution and do not wait for the response at all. When the time comes, Lambda still will call the handler function synchronously.

Answer 2

Don't use run() method and call run_until_complete()

import json
import asyncio


async def my_async_method():
    await some_async_functionality()


def lambda_handler(event, context):
    loop = asyncio.get_event_loop()    
    result = loop.run_until_complete(my_async_method())
    return {
        'statusCode': 200,
        'body': json.dumps('Hello Lambda')
    }