3

I'm querying a DynamoDB table using the hash key. Each record in the table is uniquely identified by a hash key and a range key

DynamoDBMapper mapper;
....
MyClass myClass = new MyClass();
myClass.setHashKey(hashKey);
DynamoDBQueryExpression<MyClass> queryExpression = new DynamoDBQueryExpression<MyClass>()
                                                 .withHashKeyValues(myClass);
PaginatedQueryList<MyClass> entries = mapper.query(MyClass.class, queryExpression);
//Work with the elements of entries

When the result set is more than 1MB, how can I retrieve the rest. I cannot find any method to get the LastEvaluatedKey as mentioned in the docs.

Thiyagu
  • 17,362
  • 5
  • 42
  • 79
  • Have you verified that you aren't getting all the results in your PaginatedQueryList? I think that class should be handling the implementation details of getting all the data for you. – Mark B Apr 05 '16 at 16:21
  • @MarkB No I haven't verified it. But from the [docs](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API_Query_v20111205.html) it was said that if the result is more than 1MB the query stops and we must make subsequent calls – Thiyagu Apr 05 '16 at 16:24
  • @MarkB So can I just iterate through the `entries` and it will have all the records for the hash key that I queried. I was looking into the source code of PaginatedQueryList and found things related to eager and lazy loads – Thiyagu Apr 05 '16 at 16:25
  • You're looking at the docs for the low level query function, but you are using the high-level Java DynamoDB library, which is specifically returning a paginated result which says in the docs it can make multiple calls automatically to retrieve the results. – Mark B Apr 05 '16 at 16:27
  • @MarkB I was aware that the doc was for low level APIs. But in the [DynamoDBMapper docs](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBMapper.QueryScanExample.html) nothing was mentioned about the behaviour of how query results are retrieved. So I was not sure – Thiyagu Apr 05 '16 at 16:30
  • http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/dynamodbv2/datamodeling/PaginatedQueryList.html "Paginated results are loaded on demand when the user executes an operation that requires them." – Mark B Apr 05 '16 at 16:35
  • @MarkB Infact the first operation that I do is to get the size of the list. So I believe that I get the correct full size and I can just iterate through the entries. Is it right? – Thiyagu Apr 05 '16 at 16:39
  • That sounds correct based on my reading of the documentation. – Mark B Apr 05 '16 at 17:01
  • Possible duplicate of [Pagination with DynamoDBMapper Java AWS SDK](http://stackoverflow.com/questions/32788959/pagination-with-dynamodbmapper-java-aws-sdk) – mkobit Apr 08 '16 at 15:54

1 Answers1

2

AWS SDK for Dynamodb Mapper handles pagination for you. It internally queries db and when you require data for more than 1MB, it queries it again and gets data for you. If you want the complete list in one go you can use operations like size or copying the paginated result into a list which will require mapper to fetch complete result. In short, you need not worry about LastEvaluatedKey and its handling it is handled for you.

Examples,

 PaginatedQueryList<T> resultPaginatedList = dynamoDBMapper.query(getModelClass(), queryExpression);
 
 List<T> queryList = new LinkedList<>(resultPaginatedList); //--- Line1
 logger.info("Total elements found: " + queryList.size());  //--- Line2

Both line 1 and line 2 depicts operations for which mapper will fetch the complete result (by querying multiple times handled by SDK) and not just 1MB.

Hasitha Jayawardana
  • 2,326
  • 4
  • 18
  • 36
Nikita
  • 255
  • 1
  • 4
  • 10
  • 2
    True. Though kind of confusing as `AmazonDynamoDB.qurery()` API only loads 1MB, `DynamoDBMapper` does provide this magic operation. Here is the official document: https://docs.amazonaws.cn/en_us/amazondynamodb/latest/developerguide/DynamoDBMapper.Methods.html#DynamoDBMapper.Methods.query – Feng Han Aug 05 '19 at 18:46