4

I want to create a broker cluster using Kafka 0.10 API preferable with Java. As far as I have read kafka_2.11-0.10.0.0.jar do support creating broker using :

import kafka.cluster.Broker;
import kafka.cluster.Cluster;

But I can't find any documentation for doing so. I recently read [1], which tell how to create a topic using Kafka API in Java. Can we do similar things to create broker cluster, update partition, migrate existing data/partition to new broker (as these new broker will not automatically be assigned any data partitions, so unless partitions are moved to them they won't be doing any work [2])

[1] How Can we create a topic in Kafka from the IDE using API

[2] https://kafka.apache.org/0100/ops.html#basic_ops_cluster_expansion

Community
  • 1
  • 1
gourxb
  • 325
  • 1
  • 5
  • 13

1 Answers1

-1

I have some sample code which you may find useful.

For creating a broker, take a look at KafkaTestServer. It's really for simpler testing, so it does not create a cluster, just a single broker, but it should not be difficult to extend.

Once I built in the ability to create/query/delete topics into the test server, I created a standalone admin client for doing the same against other servers, so if you are already creating a broker cluster, you should be able to use the code to maintain topics on it. Take a look at KafkaAdminClient.

The admin client is basically a pure java wrapper around the scala kafka.admin.AdminUtils class so it handles all the scala <--> java conversions under the covers.

Nicholas
  • 15,916
  • 4
  • 42
  • 66