Stack Overflow
Stack Overflow

Questions tagged [rocksdb]

RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads.

About

RocksDB is an embeddable persistent key-value store for fast storage. RocksDB can also be the foundation for a client-server database but our current focus is on embedded workloads.

RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation.

Links

474 questions
54
votes
6 answers

Redis vs RocksDB

I have read about Redis and RocksDB, I don't get the advantages of Redis over RocksDB. I know that Redis is all in-memory and RocksDB is in-memory and uses flash storage. If all data fits in-memory, which one should I choose? do they have the same…
Guille
  • 2,248
  • 3
  • 24
  • 42
33
votes
1 answer

Why Apache Kafka Streams uses RocksDB and if how is it possible to change it?

During investigation within new features in Apache Kafka 0.9 and 0.10, we had used KStreams and KTables. There is an interesting fact that Kafka uses RocksDB internally. See Introducing Kafka Streams: Stream Processing Made Simple. RocksDB is not…
Seweryn Habdank-Wojewódzki
  • 2,037
  • 4
  • 30
  • 51
14
votes
3 answers

How could WAL （write ahead log） have better performance than write directly to disk?

The WAL (Write-Ahead Log) technology has been used in many systems. The mechanism of a WAL is that when a client writes data, the system does two things: Write a log to disk and return to the client Write the data to disk, cache or memory…
Kramer Li
  • 2,284
  • 5
  • 27
  • 55
12
votes
2 answers

rocksdb out of memory

I'm trying to find out why my kafka-streams application runs out of memory. I already found out that rocksDB is consuming lots of native memory and I tried to restrict it with the following configuration: # put index and filter blocks in blockCache…
D-rk
  • 5,513
  • 1
  • 37
  • 55
12
votes
2 answers

In-memory vs persistent state stores in Kafka Streams?

I've read the stateful stream processing overview and if I understand correctly, one of the main reasons why the RocksDB is being used as a default implementation of the key value store is a fact, that unlike in-memory collections, it can handle…
Dth
  • 1,916
  • 3
  • 23
  • 34
11
votes
4 answers

Alternative to openjdk:8-alpine for Kafka Streams

I am using openjdk:8-alpine for deploying Kafka Streams application. I am using Windowing and it crashes with below error: Exception in thread "app-4a382bdc55ae-StreamThread-1" java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni94709417646402513.so:…
mukesh210
  • 2,792
  • 2
  • 19
  • 41
10
votes
1 answer

Efficiently store large list structure in RocksDB so that the data can be retrieved in pages

Description: RocksDB is a key-value storage so we can simply serialise the list of objects & store the value corresponding to a key. This would be ok if the data in the list is small enough. But if the list is large and ever increasing in size then…
Pinank Lakhani
  • 1,109
  • 2
  • 11
  • 31
10
votes
1 answer

Integrate key-value database with Spark

I'm having trouble understanding how Spark interacts with storage. I would like to make a Spark cluster that fetches data from a RocksDB database (or any other key-value store). However, at this moment, the best I can do is fetch the whole dataset…
PablodeAcero
  • 399
  • 8
  • 20
10
votes
1 answer

Where to find ldb admin tool for RocksDB?

I read about RocksDB admin tool "ldb" on https://github.com/facebook/rocksdb/wiki/Administration-and-Data-Access-Tool But I could only find the ldb repo for leveldb: https://github.com/0x00A/ldb After compiling ldb, I cannot run either command from…
James Yu
  • 399
  • 6
  • 22
10
votes
2 answers

Getting total number of key-value pairs in RocksDB

Is it possible to efficiently get the number of key-value pairs stored in a RocksDB key-value store? I have looked through the wiki, and haven't seen anything discussing this topic thus far. Is such an operation even possible?
therealrootuser
  • 10,215
  • 7
  • 31
  • 46
9
votes
6 answers

UnsatisfiedLinkError on Lib rocks DB dll when developing with Kafka Streams

I'm writing a Kafka Streams application on my development Windows machine. If I try to use the leftJoin and branch features of Kafka Streams I get the error below when executing the jar application: Exception in thread "StreamThread-1"…
gvdm
  • 3,006
  • 5
  • 35
  • 73
8
votes
1 answer

Backing up a running rocksdb-instance

I would like to backup a running rocksdb-instance to a location on the same disk in a way that is safe, and without interrupting processing during the backup. I have read: Rocksdb Backup Instructions Checkpoints Documentation Documentation in…
midor
  • 5,487
  • 2
  • 23
  • 52
8
votes
3 answers

How to install RocksDB into ubuntu?

RocksDB is a key / value database created by facebook and works very well, but it lacks documentation on how to install the production version.
Helmut Kemper
  • 662
  • 1
  • 6
  • 15
6
votes
1 answer

Performance issue in Golang's key-value store (Badger DB)

In badgerDB, we have billions of keys of type string and values of type HashValueList. In our use case length of HashValueList might be in millions. We have to encode key and value in []byte before inserting into BadgerDb; we are using the…
Vishal Jangid
  • 358
  • 1
  • 10
6
votes
1 answer

Reverse Indexing and Data modeling in Key-Value store

I am new to key-value stores. My objective is to use an embedded key-value store to keep the persistent data model. The data model comprises of few related tables if designed with conventional RDBMS. I was checking a medium article on modeling a…
Neel Basu
  • 12,638
  • 12
  • 82
  • 146
1
2 3
…
31 32