Dynamo Local from Node-aws: all operations fail "Cannot do operations on a non-existent table"

Question

I have a local dynamo-db running. I have set up my tables using the JavaScript console and they list OK from there.

I can also put and get items to my tables from the JavaScript console:

var params = { TableName:"environmentId", Item: { environmentId: {"S":"a4fe1736-98cf-4560-bcf4-cc927730dd1b"} }};
dynamodb.putItem(params, function(err, data) {
    console.log("put : err was " + JSON.stringify(err) + " and data is " + JSON.stringify(data));
});

prints put : err was null and data is {} which I'm assuming is "success" because

params = { "Key":{"environmentId":{"S":"a4fe1736-98cf-45e0-bcf4-cc927730dd1b"}},"TableName":"environmentId"}
dynamodb.getItem(params, function(err, data) {
    console.log("get : err was " + JSON.stringify(err) + " and data is " + JSON.stringify(data));
});

prints get : err was null and data is {"Item":{"environmentId":{"S":"a4fe1736-98cf-45e0-bcf4-cc927730dd1b"}}} i.e. it retrieves the object I just put to the table.

However, if it fire up the node REPL and type:

var AWS = require('aws-sdk');
AWS.config.loadFromPath("./config/credentials.js");
endpoint = new AWS.Endpoint("http://localhost:8000");
var dynamoOpts = {apiVersion: '2012-08-10', 'endpoint':endpoint};
var dynamodb = new AWS.DynamoDB(dynamoOpts);
var params = { TableName:"environmentId", Item: { environmentId: {"S":"a4fe1736-98cf-4560-bcf4-cc927730dd1b"} }};
dynamodb.putItem(params, function(err, data) {
    console.log("put : err was " + JSON.stringify(err) + " and data is " + JSON.stringify(data));
}

I get a resource not found error:

{ "message":"Cannot do operations on a non-existent table",
    "code":"ResourceNotFoundException",
    "time":"2015-04 10T10:01:26.319Z",
    "statusCode":400,
    "retryable":false,
    "retryDelay":0
}

The ASW.request object returned from the putCommand has the correct endpoint:

{ protocol: 'http:',
    host: 'localhost:8000',
    port: 8000,
    hostname: 'localhost',
    pathname: '/',
    // etc.

The same thing happens from my Node app however the same code connecting to the real AWS hosted dynamo works.

this could be tagged with other languages too, same behavior for .net. — Brett Bim, Dec 19 '19 at 23:38

score 78 · Accepted Answer · answered Apr 10 '15 at 11:13

78

The problem is that the JavaScript console and your app use different profiles (credential and region) and therefore DynamoDB local will create separate database files for them. By using the -sharedDb flag when starting the local DynamoDB, a single database file will be shared for all clients.

From the doc:

-sharedDb — DynamoDB Local will use a single database file, instead of using separate files for each credential and region. If you specify -sharedDb, all DynamoDB Local clients will interact with the same set of tables regardless of their region and credential configuration.

answered Apr 10 '15 at 11:13

ootterskog

806
7
6

1

Thanks. The fact that the JavaScript console would use a different db was not at all clear from that doc line for me. – Rog Apr 10 '15 at 11:55
1

You can use a docker container with volumes setted - `docker run -d -p 8000:8000 -v /tmp/data:/data/ dwmkerr/dynamodb -dbPath /data/` [run-amazon-dynamodb-locally-with-docker](http://www.dwmkerr.com/run-amazon-dynamodb-locally-with-docker/) **Note**: You need to setup a *Access Key Id* on Dynamo JS shell to: [DynamoDB Local loses tables?](https://forums.aws.amazon.com/message.jspa?messageID=553723) – Richard Lee Jul 16 '17 at 04:26
1

There is an official docker image available from Amazon now on Docker hub. https://hub.docker.com/r/amazon/dynamodb-local/ – wassgren Sep 08 '18 at 14:54
Thank you so much! I wish I saw this earlier – gigi2 Mar 04 '21 at 14:54
I personally had control over everything, so it was better for me to keep the in-memory tables and just make sure that regions and credentials were identical in all test fixtures. Though, I did have to run tcpdump on port 8000 before I found the last set of bad credentials. – vastlysuperiorman Dec 10 '21 at 17:27

score 49 · Answer 2 · answered Dec 06 '18 at 19:35

49

Those who are using the official DynamoDB Local Docker image should use this line to start it to enable sharedDb:

docker run -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -inMemory -sharedDb

The original ENTRYPOINT and CMD used by the image can be seen in docker inspect amazon/dynamodb-local output and are:

"Entrypoint": [
    "java"
]
"Cmd": [
    "-jar",
    "DynamoDBLocal.jar",
    "-inMemory"
]

So we basically need to copy them and add -sharedDb.

answered Dec 06 '18 at 19:35

madhead

31,729
16
153
201

9

dear god you are a life saver! – Jason May 15 '19 at 04:44
7

well what can i say, i owe you like 40 hours of life – dz902 Nov 30 '19 at 17:32
I am facing this error 'Unrecognized option: -inMemory ' – S A Jun 22 '20 at 18:57
Alternatively, make sure all your clients use the same access key ID and the right region. – Shannon Dec 14 '21 at 19:21

score 1 · Answer 3 · answered Nov 06 '21 at 01:18

If you are using CircleCI, here is what your executor should look like:

# using service containers on CircleCI
# https://circleci.com/docs/2.0/databases/
executors:
  with-dynamodb:
    docker:
      # image used to install source code,
      # run our server and run Cypress tests
      - image: cypress/base:14.17.3

      # image used to run Dynamodb in a separate container
      - image: amazon/dynamodb-local:1.17.0
        # HERE IS THE IMPORTANT PART
        command: ["-jar", "DynamoDBLocal.jar", "-inMemory", "-sharedDb"]

The command option allows you to override the default CMD in the Dockerfile, pointed out by @madhead.

Dynamo Local from Node-aws: all operations fail "Cannot do operations on a non-existent table"

3 Answers3