0

I have a use case, in which my data is stored in DynamoDB with hashkey as UniqueID and range key as Date. The same data is also present in simple storage service of Amazon(S3). I want to search all the data based on time range. I want this to be fast enough. I can think of the following possible approaches: - Scrap the full S3 and sort them based on time(it is not satisfying my latency requirements) -Using DynamoDB scan filters will not help, as they scan the whole table. Consider data to be of large amount.

Requirements : fast(can get the result in less than 1 minute) , do not access a large amount of data, can't use any other DB source

Community
  • 1
  • 1
Nitish
  • 31
  • 1
  • 4
  • I am sorry, but yes. I want possible approach to search the data without using scan filters in DynamoDB. – Nitish Oct 15 '13 at 18:54
  • Try making it more specific, show us what you've tried so far and why it hasn't worked, etc. – Scott Solmer Oct 15 '13 at 19:26
  • My answer to a similar question may be useful: http://stackoverflow.com/a/19385134/929047 - we are struggling with some of the same issues. – John Kelvie Oct 15 '13 at 22:47

2 Answers2

2

I think AWS Elasticsearch might be the answer to your problem. DynamoDB is now integrated with Elasticsearch, enabling you to perform full-text queries on your data.

Elasticsearch is a popular open source search and analytics engine designed to simplify real-time search and big data analytics.

Elasticsearch integration is easy with the new Amazon DynamoDB Logstash Plugin.

Chaitanya
  • 1,440
  • 1
  • 14
  • 26
-1

You should use Query instead of Scan, see http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html.

yegor256
  • 102,010
  • 123
  • 446
  • 597