4

For my DynamoDB table, I currently have a schema like this:

Partition key - Unique ID, so every item has a completely unique ID

Sort key - none

Attribute - JSON that contains some values

Now, I want to add a new field that will be required for every item and will indicate the specific region (e.g. NA-1, NA-2, JP-1, and so on) and I want to be able to do queries on just this field. For example, I might want to perform a query on my table to retrieve all items with the region NA-1.

My question is should I make this field a GSI? I'm new to DynamoDB so I've been researching online and it seems that using a GSI is preferred when that field may only be present for select items in the table, but my field will be required for every item, so I think using a GSI is not an option.

The other possible option I've seen is performing a scan operation and using a filter expression, but from what I've seen, that's a costly operation because DynamoDB has to look at the entire table part-by-part and then filter afterwards. My table isn't very big right now, but it may become quite large in the future, so I would like a scalable option.

TL;DR Is there someway I can add a mandatory regionID field to my table and perform efficient queries on it? What are some good options I should look into?

EMS
  • 71
  • 3
  • How many records (items) do you think you're going to be adding to the table and whats the size of each item? – dellyjm Aug 03 '20 at 01:31

3 Answers3

1

Yeah, a GSI might not be the best fit here. Maybe you can somehow make it part of the partition key?

Kawaiiii
  • 157
  • 1
  • 8
0

Yes. Perform 2 writes on the table. First row will be what you are currently writing, and the second row will have your region as the partition key. Do not forget use transactions as it is possile that one of the writes does not succeed.

While you can use GSI, you have to realize that it is eventual consistent. It will take some time to update it and you might get inconsistent data if you query soon enough after writing.

techriften
  • 421
  • 2
  • 16
0

DynamoDB is a distributed data-store i.e. it stores the data not in a single server but does partitions using the provided partition key (PK). This means your data is spread across multiple servers and brings the limitation that you can query a single partition at a time.

Coming back to your query pattern,

retrieve all items with the region X

You need to add region-id as an attribute in the main table and make it part of the GSI. Do note that to avoid conflicts you need to make the GSI SK a composite SK.
I would recommend using <region>#<unique-id>

This way you can query the GSI like,

where BEGINS_WITH ('X', SK)

Also, if any of your entry moves to a new region or a new entry is created in a region, it will automatically reflect in the GSI and your query results

swayamraina
  • 2,958
  • 26
  • 28