How to use RocksDB tailing iterator?

Question

I am using RocksDB Java JNI and would like to get new entries as they are added to the RocksDB.

Thread t = new Thread(() -> {
            for (int i = 0; i < 1000; i++) {
                try {
                    System.out.println("Putting " + i);
                    rocksDB.put(("key " + i).getBytes(), ("value " + i).getBytes());
                    Thread.sleep(100);
                } catch (InterruptedException | RocksDBException e) {
                    e.printStackTrace();
                }
            }
        }, "Putting thread");
        t.start();

       Thread.sleep(1000); // wait for sometime

       ReadOptions readOptions = new ReadOptions();
       readOptions.setTailing(true);
       try (RocksIterator rocksIterator = rocksDB.newIterator(readOptions)) {
            for (rocksIterator.seekToFirst(); rocksIterator.isValid(); rocksIterator.next()) {
                System.out.println(new String(rocksIterator.key()) + " = " + new String(rocksIterator.value()));
            }
        }
        t.join();

Here, I suppose it is creating a snapshot at that instant of time (i.e. after 1 sec) and only those elements that are added are getting printed. I expected the tailing iterator should be blocking because because new entries will be added.

Is there any example on how to use tailing iterator in RocksDB?

score 0 · Answer 1 · answered Sep 11 '20 at 20:06

The tailing iterator is not blocking, it only means the iterator can get new update after creation. Versus a normal iterator take a snapshot of current data: new data added after the iterator creation, won't be included.

auto trailing_it = rocksDB.newIterator(readOptions);
auto normal_it = rocksDB.newIterator(new ReadOptions());
rocksDB.put("new_data", "added_after_iterator_creation");
ASSERT(trailing_it.isValid());  // iterator has the new added data
ASSERT(!normal_it.isValid());   // doesn't have the new added data

For the implementation detail: a normal iterator takes a snapshot of the data with the current sequence number, new data won't be included as they have larger sequence number. Trailing iterator uses a MaxSequenceNumber, which will include any new added data.

How to use RocksDB tailing iterator?

1 Answers1