218

It's widely mentioned that Redis is "Blazing Fast" and mongoDB is fast too. But, I'm having trouble finding actual numbers comparing the results of the two. Given similar configurations, features and operations (and maybe showing how the factor changes with different configurations and operations), etc, is Redis 10x faster?, 2x faster?, 5x faster?

I'm ONLY speaking of performance. I understand that mongoDB is a different tool and has a richer feature set. This is not the "Is mongoDB better than Redis" debate. I'm asking, by what margin does Redis outperform mongoDB?

At this point, even cheap benchmarks are better than no benchmarks.

Chris Seymour
  • 83,387
  • 30
  • 160
  • 202
Homer6
  • 15,034
  • 11
  • 61
  • 81
  • 2
    In general, caring about the difference between 5,000 ops/sec and 10,000 ops/sec is often a case of premature optimization. That said, it's still an interesting answer :) – Kevin Sep 22 '13 at 22:17

7 Answers7

248

Rough results from the following benchmark: 2x write, 3x read.

Here's a simple benchmark in python you can adapt to your purposes, I was looking at how well each would perform simply setting/retrieving values:

#!/usr/bin/env python2.7
import sys, time
from pymongo import Connection
import redis

# connect to redis & mongodb
redis = redis.Redis()
mongo = Connection().test
collection = mongo['test']
collection.ensure_index('key', unique=True)

def mongo_set(data):
    for k, v in data.iteritems():
        collection.insert({'key': k, 'value': v})

def mongo_get(data):
    for k in data.iterkeys():
        val = collection.find_one({'key': k}, fields=('value',)).get('value')

def redis_set(data):
    for k, v in data.iteritems():
        redis.set(k, v)

def redis_get(data):
    for k in data.iterkeys():
        val = redis.get(k)

def do_tests(num, tests):
    # setup dict with key/values to retrieve
    data = {'key' + str(i): 'val' + str(i)*100 for i in range(num)}
    # run tests
    for test in tests:
        start = time.time()
        test(data)
        elapsed = time.time() - start
        print "Completed %s: %d ops in %.2f seconds : %.1f ops/sec" % (test.__name__, num, elapsed, num / elapsed)

if __name__ == '__main__':
    num = 1000 if len(sys.argv) == 1 else int(sys.argv[1])
    tests = [mongo_set, mongo_get, redis_set, redis_get] # order of tests is significant here!
    do_tests(num, tests)

Results for with mongodb 1.8.1 and redis 2.2.5 and latest pymongo/redis-py:

$ ./cache_benchmark.py 10000
Completed mongo_set: 10000 ops in 1.40 seconds : 7167.6 ops/sec
Completed mongo_get: 10000 ops in 2.38 seconds : 4206.2 ops/sec
Completed redis_set: 10000 ops in 0.78 seconds : 12752.6 ops/sec
Completed redis_get: 10000 ops in 0.89 seconds : 11277.0 ops/sec

Take the results with a grain of salt of course! If you are programming in another language, using other clients/different implementations, etc your results will vary wildy. Not to mention your usage will be completely different! Your best bet is to benchmark them yourself, in precisely the manner you are intending to use them. As a corollary you'll probably figure out the best way to make use of each. Always benchmark for yourself!

Homer6
  • 15,034
  • 11
  • 61
  • 81
Zach Kelling
  • 52,505
  • 13
  • 109
  • 108
  • Thank you very much... this gives me the ballpark answer I was looking for... 2x write, 3x read... perfect... of course, for specific problems, it'll be different, but this is a rough estimate... tyvm – Homer6 May 03 '11 at 22:19
  • 3
    It's worth commenting that MongoDB and Redis have different persistence structures, and that Redis only supports a data schema that is able to fit in memory. Though ram is cheap, if you need to use/store more than 12-16GB of data, I'd see what your server options look like. – Tracker1 Jul 25 '12 at 21:46
  • Hence the "I understand that mongoDB is a different tool". Any number of factors can change the access time for a given application. This was just meant to be a simple benchmark on the most fundamental of data structures. If you'd like to contribute a more exhaustive benchmark, I'm sure others would benefit from it. Please link to it, if you do. Thx. – Homer6 Sep 04 '12 at 02:30
  • 57
    @sivann this post goes from no benchmarks to a clearly stated "rough" benchmark. Don't be a troll with the "benchmarks are misleading" nonsense. Of course different conditions can alter the results. Contribute back and submit your own benchmarks that test your case and link from this post instead, then we'll all benefit from your "tested" opinion. – Homer6 Jan 07 '13 at 18:38
  • 1
    @Homer6 dear homer6, I admit I was too harsh, sorry about that. But the net is full of benchmarks with the default DB configurations and the modern DB manufacturers take advantage of that by shipping their products with all the sane options tweaked for performance rather than safety. Imagine postgres shipping with [synchronous_commit=off](http://www.postgresql.org/docs/9.2/static/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT), that would be ridiculous for them since that would disable durability. Doing something like that, actually benchmarks the memory bus and network not the DB. – sivann Jan 08 '13 at 09:58
  • 2
    @sivann The default (shipped) configuration is what this benchmark was testing. IMHO, the default configuration determines which side of the fsync fence a package sits on. For Redis, it's advertised as a memory server that urges people to use other alternatives when the database is larger than the total system memory. For MongoDB, it's advertised as a database. Postgres would never turn off fsync because they're clearly in the persistence camp. Most people don't modify the configurations, so this benchmark is somewhat accurate for those cases. – Homer6 Jan 08 '13 at 22:09
  • For more info: http://redis.io/topics/persistence or http://oldblog.antirez.com/post/redis-persistence-demystified.html – Homer6 Jan 08 '13 at 22:10
  • 4
    I agree with @sivann, the benchmark you posted is *fatally* flawed. MongoDB is multi-threaded and Redis is not. If your benchmark was multi-threaded you would see that MongoDb actually has higher throughput on a multi-core machine. – ColinM Jan 17 '13 at 18:49
  • 2
    @Homer6 even for memory-oriented DB, you should test with [WriteConcern](http://docs.mongodb.org/manual/core/write-operations/#write-concern) enabled (disabled by default). Testing without is really nonsense for any kind of benchmark. Similar for reddis. DBs which do not sync on disk all transactions, maintain safety by replicating the data to at least 2 servers. That means your writes don't wait for a disk sync, but for network replication before returning. Not waiting for errors is something never done on prod. like not detecting if the network cable is connected when writing to the network. – sivann Jan 17 '13 at 22:05
  • 1
    The default changed. http://docs.mongodb.org/manual/release-notes/drivers-write-concern/ – sivann Jan 17 '13 at 22:11
  • Redis results would be much better if you use hiredis module. For python code evaluation is better to use timeit module. Regards. – Marek Marczak Dec 10 '17 at 10:25
  • This benchmarks looks like it assumes both redis and mongo are running on the local machine. But in a typical deployment, redis would be running in memory locally while mongo would have it's own instance. That means there would normally be network latency between the local instance and mongo that doesn't seem to be measured in this benchmark... – Ned Bingham Sep 16 '21 at 15:03
18

Please check this post about Redis and MongoDB insertion performance analysis:

Up to 5000 entries mongodb $push is faster even when compared to Redis RPUSH, then it becames incredibly slow, probably the mongodb array type has linear insertion time and so it becomes slower and slower. mongodb might gain a bit of performances by exposing a constant time insertion list type, but even with the linear time array type (which can guarantee constant time look-up) it has its applications for small sets of data.

andyg0808
  • 1,367
  • 8
  • 18
Andrei Andrushkevich
  • 9,905
  • 4
  • 31
  • 42
15

Good and simple benchmark

I tried to recalculate the results again using the current versions of redis(2.6.16) and mongo(2.4.8) and here's the result

Completed mongo_set: 100000 ops in 5.23 seconds : 19134.6 ops/sec
Completed mongo_get: 100000 ops in 36.98 seconds : 2703.9 ops/sec
Completed redis_set: 100000 ops in 6.50 seconds : 15389.4 ops/sec
Completed redis_get: 100000 ops in 5.59 seconds : 17896.3 ops/sec

Also this blog post compares both of them but using node.js. It shows the effect of increasing number of entries in the database along with time.

Tareq Salah
  • 3,720
  • 4
  • 34
  • 48
8

Numbers are going to be hard to find as the two are not quite in the same space. The general answer is that Redis 10 - 30% faster when the data set fits within working memory of a single machine. Once that amount of data is exceeded, Redis fails. Mongo will slow down at an amount which depends on the type of load. For an insert only type of load one user recently reported a slowdown of 6 to 7 orders of magnitude (10,000 to 100,000 times) but that report also admitted that there were configuration issues, and that this was a very atypical working load. Normal read heavy loads anecdotally slow by about 10X when some of the data must be read from disk.

Conclusion: Redis will be faster but not by a whole lot.

John F. Miller
  • 26,961
  • 10
  • 71
  • 121
7

Here is an excellent article about session performance in the Tornado framework about 1 year old. It has a comparison between a few different implementations, of which Redis and MongoDB are included. The graph in the article states that Redis is behind MongoDB by about 10% in this specific use case.

Redis comes with a built in benchmark that will analyze the performance of the machine you are on. There is a ton of raw data from it at the Benchmark wiki for Redis. But you might have to look around a bit for Mongo. Like here, here, and some random polish numbers (but it gives you a starting point for running some MongoDB benchmarks yourself).

I believe the best solution to this problem is to perform the tests yourself in the types of situations you expect.

zed_0xff
  • 32,417
  • 7
  • 53
  • 72
mistagrooves
  • 2,337
  • 15
  • 16
  • The Tornado benchmarks align well with my own tests in using Redis and MongoDb as a Zend_Cache backend. The richer functionality of MongoDb lets you use fewer requests and the multi-threaded design scales far better than a single Redis process which is not multi-threaded. Conclusion is that MongoDb scales higher. Also, Redis no longer supports virtual memory. – ColinM Jan 17 '13 at 18:56
3

In my case, what has been a determining factor in performance comparison, is the MongoDb WriteConcern that is used. Most mongo drivers nowadays will set the default WriteConcern to ACKNOWLEDGED which means 'written to RAM' (Mongo2.6.3-WriteConcern), in that regards, it was very comparable to redis for most write operations.

But the reality is depending on your application needs and production environment setup, you may want to change this concern to WriteConcern.JOURNALED (written to oplog) or WriteConcern.FSYNCED (written to disk) or even written to replica sets (back-ups) if it is needed.

Then you may start seeing some performance decrease. Other important factors also include, how optimized your data access patterns are, index miss % (see mongostat) and indexes in general.

schwarz
  • 193
  • 2
  • 11
0

I think that the 2-3X on the shown benchmark are misleading, since if you it also depends on the hardware you run it on - from my experience, the 'stronger' the machine is, the bigger the gap (in favor of Redis) will be, probably by the fact that the benchmark hits the memory bounds limit pretty fast.

As for the memory capacity - this is partially true, since there are also ways to go around that, there are (commercial) products that writes back Redis data to disk, and also cluster (multi-sharded) solutions that overcome the memory-size limitation.

Elior Malul
  • 683
  • 6
  • 8