10

Is it possible to efficiently get the number of key-value pairs stored in a RocksDB key-value store?

I have looked through the wiki, and haven't seen anything discussing this topic thus far. Is such an operation even possible?

therealrootuser
  • 10,215
  • 7
  • 31
  • 46

2 Answers2

14

Codewisely, you could use db->GetProperty("rocksdb.estimate-num-keys", &num) to obtain the estimated number of keys stored in a rocksdb.

Another option is to use the sst_dump tool with --show_properties argument to get the number of entries, although the result would be per file basis. For example, the following command will show the properties of each SST file under the specified rocksdb directory:

sst_dump --file=/tmp/rocksdbtest-691931916/dbbench --show_properties --command=none

And here's the sample output:

Process /tmp/rocksdbtest-691931916/dbbench/000005.sst
Sst file format: block-based
Table Properties:
------------------------------
  # data blocks: 845
  # entries: 27857
  raw key size: 668568
  raw average key size: 24.000000
  raw value size: 2785700
  raw average value size: 100.000000
  data block size: 3381885
  index block size: 28473
  filter block size: 0
  (estimated) table size: 3410358
  filter policy name: N/A
  # deleted keys: 0
Process /tmp/rocksdbtest-691931916/dbbench/000008.sst
Sst file format: block-based
Table Properties:
------------------------------
  # data blocks: 845
  # entries: 27880
  raw key size: 669120
...

Combine with some shell commands, you will be able to get the total number of entries:

sst_dump --file=/tmp/rocksdbtest-691931916/dbbench --show_properties --command=none | grep entries | cut -c 14- | awk '{x+=$0}END{print "total number of entries: " x}'

And this will generate the following output:

total number of entries: 111507
keelar
  • 5,814
  • 7
  • 40
  • 79
2

There is no way to get the count exactly. But in rocksdb 3.4 which released recently, it expose an way to get an estimate count for keys, you can try it.

https://github.com/facebook/rocksdb/releases

yinqiwen
  • 604
  • 6
  • 6
  • It's been so long since 3.4 has released . Want to check, Is there any rocksdb api which can give number of (key-value) present between given slice range? – amitwdh Feb 17 '21 at 03:42