3

After going through couple of resources on Google and stack overflow(mentioned below) , I have got high level understanding when to use what but got couple of questions too

My Understanding :

  1. When used as pure in-memory memory databases both have comparable performance. But for big data where complete complete dataset can not be fit in memory or even if it can be fit (but it increases the cost), AS(aerospike) can be the good fit as it provides the mode where indexes can be kept in memory and data in SSD. I believe performance will be bit degraded(compared to completely in memory db though the way AS handles the read/write from SSD , it makes it fast then traditional disk I/O) but saves the cost and provide performance then complete data on disk. So when complete data can be fit in memory both can be equally good but when memory is constraint AS can be good case. Is that right ?

  2. Also it is said that AS provided rich and easy to set up clustering feature whereas some of clustering features in redis needs to be handled at application. Is it still hold good or it was true till couple of years back(I believe so as I see redis also provides clustering feature) ?

How is aerospike different from other key-value nosql databases?

What are the use cases where Redis is preferred to Aerospike?

emilly
  • 10,060
  • 33
  • 97
  • 172

1 Answers1

7

Your assumption in (1) is off, because it applies to (mostly) synthetic situations where all data fits in memory. What happens when you have a system that grows to many terabytes or even petabytes of data? Would you want to try and fit that data in a very expensive, hard to manage fully in-memory system containing many nodes? A modern machine can store a lot more SSD/NVMe drives than memory. If you look at the new i3en instance family type from Amazon EC2, the i3en.24xl has 768G of RAM and 60TB of NVMe storage (8 x 7.5TB). That kind of machine works very well with Aerospike, as it only stores the indexes in memory. A very large amount of data can be stored on a small cluster of such dense nodes, and perform exceptionally well.

Aerospike is used in the real world in clusters that have grown to hundreds of terabytes or even petabytes of data (tens to hundreds of billions of objects), serving millions of operations per-second, and still hitting sub-millisecond to single digit millisecond latencies. See https://www.aerospike.com/summit/ for several talks on that topic.

Another aspect affecting (1) is the fact that the performance of a single instance of Redis is misleading if in-reality you'll be deployed on multiple servers, each with multiple instances of Redis on them. Redis isn't a distributed database as Aerospike is - it requires application-side sharding (which becomes a bit of a clustering and horizontal scaling nightmare) or a separate proxy, which often ends up being the bottleneck. It's great that a single shard can do a million operations per-second, but if the proxy can't handle the combined throughput, and competes with shards for CPU and memory, there's more to the performance at scale picture than just in-memory versus data on SSD.

Unless you're looking at a tiny amount of objects or a small amount of data that isn't likely to grow, you should probably compare the two for yourself with a proof-of-concept test.

Ronen Botzer
  • 6,951
  • 22
  • 41
  • You said ` Redis isn't a distributed database as Aerospike is - it requires application-side sharding (which becomes a bit of a clustering and horizontal scaling nightmare) or a separate proxy, which often ends up being the bottleneck` Can we have multiple proxys on separate nodes so that single proxy does not become bottleneck ? Like mentioned at https://redis.io/topics/partitioning – emilly Jul 14 '19 at 08:28
  • Even in Aersospike I believe there will be some proxy or endpoint to which all clients must be connecting. Is n't it ? Basically for any distributed cluster I believe approach is same where client connects to some proxy(some call it front controller or receiver or master) ? – emilly Jul 14 '19 at 08:34
  • 1
    In a distributed database issues such as replication, partitioning are handled by the database cluster itself, rather than externally by the user application (sharding) or a middleware proxy. The cluster itself comes to agreement on the distribution of data to each node, senses new nodes joining or existing nodes leaving, and deals with rebalancing the data automatically. A database like MySQL, Redis or a data store like memcache are not distributed databases. They can be ad-hoc clustered by application-side sharding, or bolting on middleware like a Redis proxy. – Ronen Botzer Jul 14 '19 at 15:03
  • In Aerospike, Cassandra and others there is no proxy. In Aerospike a client can connect to any node of the cluster, and the first things it does is ask for the IPs of the other nodes, and the partition map. Knowing the partition map, a client can go directly to the correct node for any operation. It also keeps track of changes to the cluster. There is no front controller, no proxy, no name node, no 'master'. See https://www.aerospike.com/docs/architecture/data-distribution.html and https://www.aerospike.com/docs/architecture/clustering.html – Ronen Botzer Jul 14 '19 at 15:09