4

One the official site of leveldb(http://code.google.com/p/leveldb/), there is a performance report. I pasted as below.

Below is from official leveldb benchmark

Here is a performance report (with explanations) from the run of the included db_bench program. The results are somewhat noisy, but should be enough to get a ballpark performance estimate.

Setup

We use a database with a million entries. Each entry has a 16 byte key, and a 100 byte value. Values used by the benchmark compress to about half their original size. LevelDB: version 1.1

CPU: 4 x Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz

CPUCache: 4096 KB

Keys: 16 bytes each

Values: 100 bytes each (50 bytes after compression)

Entries: 1000000

Raw Size: 110.6 MB (estimated)

File Size: 62.9 MB (estimated)

Write performance

The "fill" benchmarks create a brand new database, in either sequential, or random order.

The "fillsync" benchmark flushes data from the operating system to the disk after every operation; the other write operations leave the data sitting in the operating system buffer cache for a while. The "overwrite" benchmark does random writes that update existing keys in the database.

fillseq : 1.765 micros/op; 62.7 MB/s

fillsync : 268.409 micros/op; 0.4 MB/s (10000 ops)

fillrandom : 2.460 micros/op; 45.0 MB/s

overwrite : 2.380 micros/op; 46.5 MB/s

Each "op" above corresponds to a write of a single key/value pair. I.e., a random write benchmark goes at approximately 400,000 writes per second.

Below is from My leveldb benchmark

I did some benchmark for leveldb but got write speed 100 times less than the report.

Here is my experiment settings:

  1. CPU: Intel Core2 Duo T6670 2.20GHz
  2. 3.0GB memory
  3. 32-bit Windows 7
  4. without compression
  5. options.write_buffer_size = 100MB
  6. options.block_cache = 640MB

What I did is very simple: I just put 2 million {key, value} and no reads at all. The key is a byte array which has 20 random bytes and the value is a byte array too with 100 random bytes. I constantly put newly random {key, value} for 2 million times, without any operation else.

In my experiment, I can see that the speed of writing decreases from the very beginning. The instant speed (measuring the speed of every 1024 writes) swings between 50/s to 10, 000/s. And my overall average speed of writes for 2 million pairs is around 3,000/s. The peak speed of writes is 10, 000/s.

As the report claimed that the speed of writes can be 400, 000/s, the write speed of my benchmark is 40 to 130 times slower and I am just wondering what's wrong with my benchmark.

I don't need to paste my testing codes here as it is super easy, I just have a while loop for 2 million times, and inside the loop, for every iteration, I generate a 20 bytes of key, and 100 bytes of value, and then put them to the leveldb database. I also measured the time spent on {key, value} generation, it costs 0 ms.

Can anyone help me with this? How can I achieve 400, 000/s writes speed with leveldb? What settings I should improve to?

Thanks

Moreover

I just ran the official db_bench.cc on my machie. It is 28 times slower than the report.

I think as I used their own benchmark program, the only difference between my benchmark and theirs is the machine.

Jack
  • 3,913
  • 8
  • 41
  • 66
  • The setup you used and they used are *massively* different. You have a single dual core processor, while they have *four quad* core processors, each of which is clocked faster. They likely also have *far* more than 3.0 GBs of memory in their system, although it doesn't look like the working set is big enough to be a big issue for memory... but their 4x4 CPUs would have *far* more cache on hand, which since the working set is fairly small, should provide a massive performance increase. Your system will never get anywhere near the same performance as theirs, simply because of their hardware. – Kitsune Feb 22 '12 at 15:07
  • 1
    @Kitsune, so this diff in hardwares can really produce 30 times difference on performance? – Jack Feb 22 '12 at 15:19

2 Answers2

3

You have 2 million key-value pairs and each key value pair is a total of 120 bytes, so 2 million * 120 bytes = 228 MB of data! Your cache is 640 MB so it's quite possible that all of your data is still in RAM and it never really got to disk. As Kitsune pointed out: your hardware is nowhere near as fast as the one that Google tested with and if Google had the same cache size then that could easily produce 30 times difference.

Other potential issues:

  • It's difficult to know exactly how "random" were the keys: LevelDB performs differently depending on the distribution of keys (even if it's "random").
  • 20 byte keys would be less efficient than 16 bytes keys, because they don't align as well.
  • Depending on your hard drive, your disk write speed might be slower (you check yours).

We can keep going on and on and on, but there are just too many variables to consider. If you post some code that demonstrates how your test runs, then we can recommend some optimizations so you can get better performance.

Kiril
  • 39,672
  • 31
  • 167
  • 226
  • good to see you here again. Here is the post I created in leveldb group https://groups.google.com/forum/?fromgroups#!topic/leveldb/iaVNKN_dlDg. sanjay suggested I turn on optimised mode for compiler and turn off assertion and turn on compression. Could you please give me some hints on this? – Jack Feb 23 '12 at 10:51
  • also, do you know how can I add Snappy compression into leveldb c# port? – Jack Feb 23 '12 at 11:23
  • Compiling in optimized mode means that you compile the Release version instead of Debug (you can muck with the configurations to compile in optimized mode for the Debug version, but I wouldn't bother). To compile with snappy you have to get the [Snappy source](http://code.google.com/p/snappy/) and statically link it into leveldb just like you statically linked leveldb into the C++/CLI wrapper. You also have to add the `SNAPPY` flag inside leveldb's Preprocessor Definitions: Config Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions. – Kiril Feb 23 '12 at 16:05
  • FYI, "turn on compression" means that you statically link Snappy (as described above) and then in your wrapper's Open method, you have to specify the compression `options.compression = leveldb::kSnappyCompression`. That will open the leveldb instance with Snappy compression. – Kiril Feb 23 '12 at 16:08
  • I'm not sure about the assertions tho, perhaps he means this: http://stackoverflow.com/questions/2246096/how-can-i-turn-off-assert-x-in-c However, you will have to redefine assert after the #include of any code that uses the normal definition... frankly, that's a pain! – Kiril Feb 23 '12 at 16:16
  • thanks for your suggestions. For Snappy, I have downloaded the source files and created a project in my vc. and I successfully compiled it as a dll. So, in the same solution, leveldb project is also there. then I edited leveldb project's property and "Add References" of the Snappy project. Also, I added "SNAPPY" flag for Preprocessor Definitions (it is under IntelliSense group, I am using vc express). After all these, Snappy is still NOT linked with leveldb. – Jack Feb 29 '12 at 11:22
  • @Jack, you should statically link Snappy. That means that you should change the target build of Snappy from `DLL` to `LIB`. In the comment section of [your other question](http://stackoverflow.com/questions/9292648/is-there-a-good-port-of-leveldb-for-c) I gave you instructions on how to statically link LevelDB into your C++/CLI wrapper. In your LevelDB project's configuration: add the Scnappy/include path to the Additional Include directories, add Snappy's output directory to the Additional Library Dependencies, and add snappy.lib to Additional Dependencies. – Kiril Feb 29 '12 at 14:07
1

When you run the same benchmark on completely different hardware you're bound to see some differences.

  • Your CPU is ~9x weaker 2xCores@2.2GHz vs 16xCores@2.4GHz
  • Your hard drive and the drive of the official benchmark were not mentioned (fiber NAS vs a solid state drive SSD vs a hard disk drive HDD)

Can't compare apples to oranges or apples to [unknown fruit].

Louis Ricci
  • 20,804
  • 5
  • 48
  • 62