4

I see this error continuously in the debug.log in cassandra, 

WARN  [SharedPool-Worker-2] 2018-05-16 08:33:48,585 BatchStatement.java:287 - Batch of prepared statements for [test, test1] is of size 6419, exceeding specified threshold of 5120 by 1299.

In this 

   where,
                    6419 - Input payload size (Batch)
                    5120 - Threshold size
                    1299 - Byte size above threshold value

so as per this ticket in Cassandra, https://github.com/krasserm/akka-persistence-cassandra/issues/33 I see that it is due to the increase in input payload size so I Increased the commitlog_segment_size_in_mb in cassandra.yml to 60mb and we are not facing this warning anymore.

Is this Warning harmful? Increasing the commitlog_segment_size_in_mb will it affect anything in performance?

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Harry
  • 3,072
  • 6
  • 43
  • 100

2 Answers2

6

This is not related to the commit log size directly, and I wonder why its change lead to disappearing of the warning...

The batch size threshold is controlled by batch_size_warn_threshold_in_kb parameter that is default to 5kb (5120 bytes).

You can increase this parameter to higher value, but you really need to have good reason for using batches - it would be nice to understand the context of their usage...

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • I am using this batch query for transaction, Is this warning harmful? Increasing the 'batch_size_warn_threshold_in_kb' will it impact the application? I got confused with this ticket : https://github.com/krasserm/akka-persistence-cassandra/issues/33 – Harry May 17 '18 at 11:26
  • 1
    Warning is not very harmful, but you need to control how big are you batches, so set it to reasonable value... – Alex Ott May 17 '18 at 11:30
  • 1
    I really don't understand why in this ticket, people are changing the wrong setting... – Alex Ott May 17 '18 at 11:32
  • What will be the consequence of the huge batch size ? – Harry May 17 '18 at 11:38
  • 1
    Usually huge batch size means a lot of individual statements. The consequences are really dependent on the type of batch, and data in the batch - are they for same partition, or not, etc. Check sections on good & wrong use of batches at https://docs.datastax.com/en/cql/3.3/cql/cql_using/useBatchTOC.html – Alex Ott May 17 '18 at 12:04
  • Another treat for you : https://stackoverflow.com/questions/50432926/native-transport-requests-in-cassandra Please do the needful :) – Harry May 20 '18 at 08:22
-1

commit_log_segment_size_in_mb represents your block size for commit log archiving or point-in-time backup. These are only active if you have configured archive_command or restore_command in your commitlog_archiving.properties file. Default size is 32mb.

As per Expert Apache Cassandra Administration book:

you must ensure that value of commitlog_segment_size_in_mb must be twice the value of max_mutation_size_in_kb.

you can take reference of this:

Mutation of 17076203 bytes is too large for the maxiumum size of 16777216

Payal
  • 564
  • 3
  • 12
  • Thank for responding, But as per another StackOverflow link provided, I could see that Cassandra can accept (commitlog_segment_size_in_mb / 2) times the data size in the payload. But why is that throwing threshold error for 5KB? I have commitlog_segment_size_in_mb value as 32MB, What is the significane of 5KB in it? – Harry May 17 '18 at 07:48
  • also I would like to know Is this warning harmful? Please give some light – Harry May 17 '18 at 08:06
  • The title of the question is incorrect - this message is related to completely different setting... – Alex Ott May 17 '18 at 10:17