Increase number of partitions for a topic in Java

Question

I am using name : kafka_2.12 version : 2.3.0. Based on the traffic/load I want to change the maximum partition number for a topic. Is it possible to make this kind of change once Kafka is up and can it be done by code?

Answer 1

Yes you could increase partition by code. Use AdminClient.createPartitions method.

AdminClients.createPartitions method API document

public abstract CreatePartitionsResult createPartitions(java.util.Map<java.lang.String,NewPartitions> newPartitions,CreatePartitionsOptions options)

Increase the number of partitions of the topics given as the keys of newPartitions according to the corresponding values. If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected.

This operation is not transactional so it may succeed for some topics while fail for others.

It may take several seconds after this method returns success for all the brokers to become aware that the partitions have been created. During this time, describeTopics(Collection) may not return information about the new partitions.

How to use:

public static void createPartitions(String topicName, int numPartitions) {
    Properties props = new Properties();
    props.put("bootstrap.servers","localhost:9092");
    AdminClient adminClient = AdminClient.create(props);

    Map<String, NewPartitions> newPartitionSet = new HashMap<>();
    newPartitionSet.put(topicName, NewPartitions.increaseTo(numPartitions));
    adminClient.createPartitions(newPartitionSet);
    adminClient.close();
}