Slow on fetching records from DynamoDB table

Question

I have a DynamoDB table with 151 records, table size is 666 kilobytes and average item size is 4,410.83 bytes.

This is my table schema:

uid - partition key
version - sort key
status - published
archived - boolean
... // other attributes

When I'm doing scan operation from AWS Lambda.

module.exports.searchPois = async (event) => {
  const klass = await getKlass.byEnv(process.env.STAGE, 'custom_pois')

  const status = (event.queryStringParameters && event.queryStringParameters.status) || 'published'

  const customPois = await klass.scan()
    .filter('archived').eq(false)
    .and()
    .filter('status').eq(status)
    .exec()

  return customPois
}

This request takes up 7+ seconds to fetch. I'm thinking to add a GSI so I can perform a query operation. But before I add, is it really like this slow when using scan and If I add GSI, will it fetch faster like 1-3 seconds?

Do you plan to query ddb only through scan queries? I advise against this pattern. Build specific partitions & sortKeys to execute DDB queries to fetch your elements. It will soon become a performance disaster. — zenbeni, Nov 08 '22 at 09:18
Hi @BorislavStoilov i updated my question and added the whole lambda function code — AllenC, Nov 10 '22 at 01:15
Just to share my experience (first project I did with DynamoDB): Doing a scan of a DynamoDB with 256 items in it (an item has 13 properties), using DotNet core library with object model (not document model), running in a container in ECS, takes ~300 ms (cold) - 50 ms (warm), with everything as default as it can be. — Dutchman, Feb 12 '23 at 20:45

score 2 · Accepted Answer · answered Nov 08 '22 at 09:23

Using Scan should be avoided of at all possible. For your use-case a GSI would be much more efficient and a sparse index would be even better: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-indexes-general-sparse-indexes.html

In saying that, for the small number of items you have it should not take 7seconds. This is likely caused by making infrequent requests to DynamoDB as DynamoDB relies on caching metadata to improve latency for requests, if your requests are infrequent then the metadata will no exist in cache increasing response times.

I suggest to ensure you re-use your connections, create your client outside of the Lambda event handler and ensure you keep active traffic on the table.

Slow on fetching records from DynamoDB table

1 Answers1