Dynamo Db vs Elastic Search

Question

I was just reading about elastic search and found that it indexes each and every term in the document as well as all fields. Although it has some disadvantages like it cannot provide transactions, etc. But for the application, where I only need to read data from DB and there is no write, is there any advantage to using Dynamo Db instead of Elastic Search. Earlier, I was thinking to use the Dynamo Db, but now after seeing that it indexes each and every field, so why not use Elastic Search itself. Till now, the only use case defined for my project is to search by an id. But in future, more use cases come, then it would be very difficult to add more indexes in Dynamo Db but would already be there in Elastic Search.

Can someone tell me of some advantages of Dynamo Db against Elastic Search.

Please give your suggestions.

Answer 1

I have used elasticsearch and MongoDB but not much Dynamodb. MongoDB works very well regarding indexing and strong consistency.

Few things I know about elasticsearch and DynamoDB;

elasticsearch is a search-engine, where you can search by any terms, or aggregate records based on certain criteria, but it also serves as a document-store, though was not primary purpose. And definitely great for less writes and more reads.

some elasticsearch advantages

• max size of each document it allows is 2G -> 2G at lucene level

• you can have partitions(called Shards) to distribute your documents based on _id field by default

• supports strong consistency - configurable

• supports atomic writes with versioning on each document.

• supports only document level Atomicity

• you can aggregate documents based on criterias - JSON based queries

elasticsearch disadvantages

• has no Atomicity (A in ACID) between multiple documents

• you might want to check security options, last time I used it maybe version 3, did not have good option

Dynamodb on the other hand is a datastore(technically called document-store/ Amazon's version of MongoDB).

advantages

• fully managed by AWS, as you won't have to worry about server going down or etc.

• secured by Amazon IAM roles

• you can have partitions(called Shards) to distribute your documents based on primary key on each document

• supports multi item/document level Atomicity (A in ACID)

• supports atomic writes - https://github.com/awslabs/dynamodb-transactions lock attribute on each document which refers to current transactionId

• supports Eventually Consistent and Strongly Consistent Reads :

When a document is written to DynamoDB table and receives an HTTP 200 response, all copies of the document are updated. The document will eventually be consistent across all storage locations, usually within one second or less.

When you request a strongly consistent read, DynamoDB returns a response with the most up-to-date data, reflecting the updates from all prior write operations that were successful. A strongly consistent read might not be available in the case of a network delay or outage.

But has some limitations,

• only supports max 40K writes for 1KB sized documents/sec per table = which would be 400 writes for 100K sized docs/sec (in us-east region)

  supports only 10K writes for 1KB sized docs/per table in other regions

• max 40K reads for 4KB sized documents/sec per table (in us-east region)

  supports only 10K reads for 4KB sized docs/per table in other regions

  so calculate your throughput based on your average document size and see DynamoDB fits in

• the max document/item size in dynamodb is 400KB (reference to s3 might do the trick if document size is more than 400KB, but still depends if you really want to go that route )/ MongoDB might be alternative which allows upto 16M of document.

• you can only fetch 1000 KB of documents from DynamoDB in one request

So, basically,

• desired throughput,
• ACID-compliancy (DynamoDB +1),
• each document size (elasticsearch +1, MongoDB +1 ) and
• security might be the deciding factor.

I would also consider looking into MongoDB vs DynamoDB as MongoDB is open source, has all the features other than A in Atomicity, and is also supported by AWS.