Putting to local DynamoDB table with Python boto3 times out

Question

I am attempting to programmatically put data into a locally running DynamoDB Container by triggering a Python lambda expression.

I'm trying to follow the template provided here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/GettingStarted.Python.03.html I am using the amazon/dynamodb-local you can download here: https://hub.docker.com/r/amazon/dynamodb-local

Using Ubuntu 18.04.2 LTS to run the container and lambda server AWS Sam CLI to run my Lambda api Docker Version 18.09.4 Python 3.6 (You can see this in sam logs below) Startup command for python lambda is just "sam local start-api"

First my Lambda Code

import json
import boto3

def lambda_handler(event, context):
    print("before grabbing dynamodb")
#    dynamodb = boto3.resource('dynamodb', endpoint_url="http://localhost:8000",region_name='us-west-2',AWS_ACCESS_KEY_ID='RANDOM',AWS_SECRET_ACCESS_KEY='RANDOM')
    dynamodb = boto3.resource('dynamodb', endpoint_url="http://localhost:8000")
    table = dynamodb.Table('ContactRequests')
    try:
        response = table.put_item(
            Item={
                'id': "1234",
                'name': "test user",
                'email': "testEmail@gmail.com"
                }
            )

    print("response: " + str(response))

    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "hello world"
        }),
    }

I know that I should have this table ContactRequests available at localhost:8000, because I can run this script to view my docker container dynamodb tables I have tested this with a variety of values in the boto.resource call to include the access keys, region names, and secret keys, with no improvement to result

dev@ubuntu:~/Projects$ aws dynamodb list-tables --endpoint-url http://localhost:8000
{
    "TableNames": [
        "ContactRequests"
    ]
}

I am also able to successfully hit the localhost:8000/shell that dynamodb offers

Unfortunately while running, if I hit the endpoint that triggers this method, I get a timeout that logs like so

Fetching lambci/lambda:python3.6 Docker container image......
2019-04-09 15:52:08 Mounting /home/dev/Projects/sam-app/.aws-sam/build/HelloWorldFunction as /var/task:ro inside runtime container
2019-04-09 15:52:12 Function 'HelloWorldFunction' timed out after 3 seconds
2019-04-09 15:52:13 Function returned an invalid response (must include one of: body, headers or statusCode in the response object). Response received: 
2019-04-09 15:52:13 127.0.0.1 - - [09/Apr/2019 15:52:13] "GET /hello HTTP/1.1" 502 -

Notice that none of my print methods are being triggered, if I remove the call to table.put, then the print methods are successfully called.

I've seen similar questions on Stack Overflow such as this lambda python dynamodb write gets timeout error that state that the problem is I am using a local db, but shouldn't I still be able to write to a local db with boto3, if I point it to my locally running dynamodb instance?

score 3 · Answer 1 · answered Apr 10 '19 at 08:38

3

Your Docker container running the Lambda function can't reach the DynamoDB at 127.0.0.1. Try instead the name of your DynamoDB local docker container as the host name for the endpoint:

dynamodb = boto3.resource('dynamodb', endpoint_url="http://<DynamoDB_LOCAL_NAME>:8000")

You can use docker ps to find the <DynamoDB_LOCAL_NAME> or give it a name:

 docker run --name dynamodb amazon/dynamodb-local

and then connect:

dynamodb = boto3.resource('dynamodb', endpoint_url="http://dynamodb:8000")

answered Apr 10 '19 at 08:38

Reto Aebersold

16,306
5
55
74

I'm trying this now and still receiving the same result. I'm able to confirm the name of my docker container. I would expect that if I could set endpoint_url="https://dynamodb:8000", then I would be able to hit that url, but that url does not load either. Looking through docker documentation, I'm unable to locate much information on using the container name as a hostname. Do you have a link to docs that might mention that usage more, so I could dive a bit deeper into what I may be doing wrong? – Trent Miller Apr 10 '19 at 15:51

Trent Miller · Answer 2 · 2019-04-16T17:16:22.170

Found the solution to the problem here: connecting AWS SAM Local with dynamodb in docker The question asker noted that he saw online that he may need to connect to the same docker network using:

docker network create lambda-local

So created this network, then updated my sam command and my docker commands to use this network, like so:

docker run --name dynamodb -p 8000:8000 --network=local-lambda amazon/dynamodb-local

sam local start-api --docker-network local-lambda

After that I no longer experienced the timeout issue. I'm still working on understanding exactly why this was the issue

To be fair though, it was important that I use the dynamodb container name as the host for my boto3 resource call as well. So in the end, it was a combination of the solution above and the answer provided by "Reto Aebersold" that created the final solution

dynamodb = boto3.resource('dynamodb', endpoint_url="http://<DynamoDB_LOCAL_NAME>:8000")

Putting to local DynamoDB table with Python boto3 times out

2 Answers2

Linked