3

I have several machines processing large amounts of text data (100s of GB) that is indexed in RocksDB. The machines are for load balancing and are operating on the same data. When I add new machines, I want to copy the database over the network from an existing machine, as quickly as possible.

Is there an elegant way to make a RocksDB backup over the network? I have read https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB but this would require twice the amount of disk space: For a backup onto the local filesystem first, before it can be copied over the network. I would also have to deal with e.g. rsyncing files.

cidermole
  • 5,662
  • 1
  • 15
  • 21

1 Answers1

0

You can attach a volume to the server and make the backup to that - and then use that volume on the new node

Alternatively, you can loop over the entire DB at a particular LSN and stream it over some protocol

Asad Awadia
  • 1,417
  • 2
  • 9
  • 15