1

A memtable is created for every table or column family. There can be multiple memtables for a table but only one of them will be active. The rest will be waiting to be flushed. There are a few properties that affect a memtables size and flushing frequency. These include:

memtable_flush_writers – This is the number of threads allocated for flushing memtables to disk. This defaults to two.

memtable_heap_space_in_mb – This is the total allocated space for all memtables on an Apache Cassandra node. By default, this is one-fourth your heap size. Specifying this property results in an absolute heap size in MB as opposed to a percentage of the total JVM heap.

memtable_cleanup_threshold – A percentage of your total available memtable space that will trigger a memtable cleanup. memtable_cleanup_threshold defaults to 1 / (memtable_flush_writers + 1). By default this is essentially 33% of your memtable_heap_space_in_mb.

A scheduled cleanup results in flushing of the table/column family that occupies the largest portion of memtable space. This keeps happening till your available memtable memory drops below the cleanup threshold.

Let assume we have an Apache Cassandra instance that has allocated 4G of space. Out of this only 3,925.5MB is available to the Java runtime. Please look at the following StackOverflow question(Why do -Xmx and Runtime.maxMemory not agree) for the reasons behind this. Of this, by default, we have 981 MB allocated towards memtable i.e. 1/4the of 3,925.5. Our memtable_cleanup_threshold is the default value i.e. 33 percent of the total memtable heap and off heap memory. In our example that comes to 327 MB. Thus when total space allocated for all memtables is greater than 327 MB a memtable clean-up is triggered. The cleanup process looks for the largest memtable and flushes that to disk.

if I am allocating 981MB for mem table and cassandra initiates a flush after 327 Mb, that means at any point of time cassandra will have max of 327 mb of active memtables...then what about (981-327)mb = 654mb mem space.What is it used for. I could sense that memtables which are in queue to be flushes occupy some portion of this 654mb, but what about the rest of the spaces, it not it being wasted??

Rupesh Mukherjee
  • 75
  • 1
  • 7
  • Essentially a dup of your question in https://stackoverflow.com/questions/45663442/cassandra-mem-table-content. The space is not allocated in a chunk and reserved or wasted, its a limit that prevents writes from OOMing your system. The spaced used can and will exceed the threshold*space. – Chris Lohfink Aug 14 '17 at 16:39

1 Answers1

0

memtable_heap_space_in_mb decides how much heap can be used for memtable. It's not mandatory to allocate all of them to memtable. If there are 327 mb for memtable, the other memory (total heap) could be used for queries or repair operations.

ouflak
  • 2,458
  • 10
  • 44
  • 49
eason
  • 1