DynamoDB : List all partition keys

Question

I want to update contains in DynamoDB, for which I need to iterate over existing partition keys present in table. Is there any way to fetch only list of partition keys using Python. Scan and Query only work on attributes of my table. Is there any way to get all partition key for table ?

score 11 · Accepted Answer · answered Feb 11 '18 at 17:24

11

If your table uses sort keys in addition to the partition keys (stated differently, if the keys are composite of partition + sort key) then the answer is: no - there is no way to query or scan for just the partition keys. To clarify, you can still scan your table with a projection that returns the keys only, but it will return each primary key multiple times, once for each item that has the same primary key with a different sort key.

If your table schema uses partition keys only (no sort key) then you can write a scan with a projection of only the primary key and therefore, get the list of partition keys as a result.

answered Feb 11 '18 at 17:24

Mike Dinescu

54,171
16
118
151

Well, as I mentioned in the answer, there is no way to query just for distinct partition keys. You have to scan to determine the keys. – Mike Dinescu Feb 13 '18 at 05:28
It worked with projection, i am able to retrieve only partition keys suing that - Thanks – WPrathamesh Feb 14 '18 at 06:34
Here there is something important to note, that you can NOT query just for partition keys either, distinct or not. – Ali_Nass Jul 08 '21 at 09:59
1

Some years after AWS has published some smart techniques how to list all partition keys in a more efficient way: https://aws.amazon.com/blogs/database/generate-a-distinct-set-of-partition-keys-for-an-amazon-dynamodb-table-efficiently/ – MrTJ Jul 21 '23 at 13:49

score 1 · Answer 2 · answered Jan 03 '23 at 12:27

Overview

To gain all of the partition keys from a table you need to use Scan which will read all of the items in the table. As you are only wanting keys returned, you can use the ProjectionExpression parameter to specify which attributes you would like to be returned.

Scan

The Scan operation returns one or more items and item attributes by accessing every item in a table or a secondary index. To have DynamoDB return fewer items, you can provide a FilterExpression operation.

ProjectionExpression

A string that identifies one or more attributes to retrieve from the specified table or index. These attributes can include scalars, sets, or elements of a JSON document. The attributes in the expression must be separated by commas.If no attribute names are specified, then all attributes will be returned. If any of the requested attributes are not found, they will not appear in the result.

Solution

My Table

pk	sk	col1	col2	col3
123	abc	data	data	data
456	def	data	data	data
789	ghi	data	data	data

Scan with ProjectionExpression

aws dynamodb scan \
    --table-name MusicCollection \
    --projection-expression "pk, sk"

Response

pk	sk
123	abc
456	def
789	ghi

score 1 · Answer 3 · answered Jun 17 '23 at 05:05

There is a way to extract the distinct list of PKs using some creative manipulation of the LastEvaluatedKey to avoid a full table scan.

Discussion and code is in this blog:

https://aws.amazon.com/blogs/database/generate-a-distinct-set-of-partition-keys-for-an-amazon-dynamodb-table-efficiently/