AWS lambda with python asyncio. Event loop closed problem?

Question

Closing the event loop in aws lambda affects future lambda runs??

• I have some aysncio python code running within an aws lambda service. The logic of the code is as follows

  def lambda_handler(event,context):
      loop = asyncio.get_event_loop()
      # perform all operations with the loop
      loop.close()
      return results
  

• If I run this once, it appears to work fine. However, if I rerun it immediately afterwards, I get an error saying Event loop closed

• Why is this happening? Shouldn't each lambda run be independent of the last? After all lambda is supposed to be stateless

Answer 1

Python 3.7+

You can use the higher level asyncio.run() that will take care of things.

def handler(event, context):
    asyncio.run(main())

async def main():
    # your async code here

This will close the loop at the end and open a new one when running lambda again. .run() is also recommended by asyncio maintainers.

Answer 2

After all lambda is supposed to be stateless

Your function is supposed to be stateless.

From https://aws.amazon.com/lambda/faqs/:

Q: What is an AWS Lambda function?

...

The code must be written in a “stateless” style i.e. it should assume there is no affinity to the underlying compute infrastructure.

...

Q: Will AWS Lambda reuse function instances?

To improve performance, AWS Lambda may choose to retain an instance of your function and reuse it to serve a subsequent request, rather than creating a new copy. To learn more about how Lambda reuses function instances, visit our documentation. Your code should not assume that this will always happen.

The current python instance is reused for performance reasons, but its reuse or non-reuse must never be relied on. So while AWS Lambda is not itself always stateless, your programming methodology should be. Hopefully that clears up your confusion as to why that is happening!

Answer 3

Add this line of the top of the code,

asyncio.set_event_loop(asyncio.new_event_loop())

so that it is global.

Alternatively,

Replace,

loop = asyncio.get_event_loop()

with,

loop = asyncio.new_event_loop()

It is happening since you already closed the loop.

That should work.

Answer 4

As of 2023-1-3, I found all existing solutions don't work within Python3.9 environment:

• new a loop: got Future <Future pending> attached to a different loop
• asyncio.run: Asyncio Event Loop is Closed
• and some other tricks seen in other answers I didn't record

I have to do this:

import asyncio

async def work(msg):
    await do_something_with(msg)

def lambda_handler(event, context):
    msg = event.get("body", "emptybody") # your msg should vary
    loop = asyncio.get_event_loop()
    if loop.is_closed(): # I don't know why
        loop = asyncio.new_event_loop()
    loop.run_until_complete(work(msg))

Only then the first run and consequent runs won't fail.

My best random guess is that Lambda sometimes starts from the top of the code, and sometimes just reuses lambda_handler, and it will close event loops sometimes