3

Need some info on how chronicle map works, is it like it keeps some key-value pairs in memory and when it overflows a particular threshold of how may value it stores then it overflows the data to disk or it is dependant on memory size, if the size of map grows above a threshold then overflow the data to disk, if so then how can it be configured, or is there any other strategy ?

leventov
  • 14,760
  • 11
  • 69
  • 98
Ambuj Jauhari
  • 1,187
  • 4
  • 26
  • 46

1 Answers1

9

Chronicle Map writes directly to memory mapped files. This is entirely off heap. If you write an entry, this entry is immediately visible to other processes on the same machine. Also if your JVM crashes, the data is not lost or corrupted.

Once the data is written there is nothing for the application to do. (Which is why it can crash without loss of data)

The amount of data which is in memory but has not been written to disk is determined by the OS. It is usually around 10% of main memory. The OS will write the data to disk over time (seconds usually)

Entries which have been written to disk use the read cache of the OS. The read cache can be the entire free memory of your system. The way to change this limit is to add more memory.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • 1
    We are using Chronicle Map. Its pretty amazing. We have around 3 million entry map and it takes only 100MB off heap which is amazing . – Makky Dec 23 '18 at 20:12
  • @Makky and adds almost nothing to your heap or GC times. Do you use the persisted version? – Peter Lawrey Dec 24 '18 at 19:12
  • Sir I am using "Create()" I think non-file version – Makky Dec 24 '18 at 19:13
  • I am introducing this in my company and we are hoping to use that across the teams will be using FileBased for distributed version. QQ Is the Chronicle Map thread safe ? – Makky Dec 24 '18 at 19:14
  • @Makky correct. If you use the other version it can be shared between JVMs and is persisted. – Peter Lawrey Dec 24 '18 at 19:14
  • I'd suggest that create some tutorials/YT as I had to do alot of trial error to get it running. PS Our knowledge of JAva is like 1% of yours – Makky Dec 24 '18 at 19:15
  • 1
    @Makky on the Chronicle Map home page https://github.com/OpenHFT/Chronicle-Map there is a link to these tutorials. https://github.com/OpenHFT/Chronicle-Map/blob/master/docs/CM_Tutorial.adoc I also suggest you have a look at the unit tests for more working examples. – Peter Lawrey Dec 24 '18 at 19:16
  • @Peter Lawrey , do you guys know how often does chronicle map/OS persist data from memory to the disk. Is this controllable? Also if a process crashes with chronicle map , is the occupied memory off-heap flushed out? Do we have to flush it manually? – howard roark Aug 18 '22 at 13:38
  • @Makky if possible could you provide some insights into the above question? – howard roark Aug 18 '22 at 13:39