kafka - proper way to set java startup params

Question

There seem to be conflicting schools of thought when it comes to setting the various heap, GC, etc java params for a Kafka process.

One group says to edit the kafka-server-start bash file

• here

Another group says to set a system var and let the kafka startup pick it up

• here

In the latest instructions from Confluent the follow 'recommendations' appear:

-Xms6g -Xmx6g -XX:MetaspaceSize=96m -XX:+UseG1GC -XX:MaxGCPauseMillis=20
   -XX:InitiatingHeapOccupancyPercent=35 -XX:G1HeapRegionSize=16M
   -XX:MinMetaspaceFreeRatio=50 -XX:MaxMetaspaceFreeRatio=80

What's the official Confluent party-line on where to set these? (I assume the defaults don't use these to accommodate smaller test machines)

Answer 1

Both of these things are essentially telling you to do the same thing, which is manually overriding the value of KAFKA_HEAP_OPTS.

It looks like this is what you are looking for from kafka-server-start.sh

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi

EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}

If you are starting this from the command line, you can call

export KAFKA_HEAP_OPTS="-Xmx2g -Xms2g"
./kafka-server-start.sh

Or, if you are editing the startup script first, you do not need to export, because the variable is used locally.

KAFKA_HEAP_OPTS="-Xmx2g -Xms2g"
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi

The benefit of the first, is that it does not require changes to kafka-source, and allows for the environment to be configured via just shell variables;

The second is convenient in that you do not need to remember to setup your environment variables before starting the server.