11

I am trying to create a kafka topic via AdminCommand using below code Source

 ZkClient zkClient = new ZkClient(kafkaHost, 10000, 10000, ZKStringSerializer$.MODULE$);
    AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());

But getting the below exception

Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)

However, I am able to create the topic using shell command .

Community
  • 1
  • 1
Count
  • 1,395
  • 2
  • 19
  • 40

3 Answers3

14

In your code,

 AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());

The fourth argument is the replication factor. So you are trying to create a topic with a name of pa_reliancepoc_telecom_usageevent with partition count of 10 and replication of 2. So two kafka brokers should be available while creating the topic. If less than two is available then you will get the following exception.

Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)

Make sure that you are running kafka cluster with two broker nodes and the two nodes should be alive while creating the topic.

To run kafka in cluster refer Step 6 in this link

Jaya Ananthram
  • 3,433
  • 1
  • 22
  • 37
  • In any case, the exception is grossly misleading and probably should read: "[...] replication factor: 2 larger than available brokers: 1 " – sandris Mar 09 '15 at 16:23
5

configure your local machine to have multiple brokers up and running in case you decide to keep your replication_factor > 1.

You can do that by simply having multiple copies of the server.properties file. e.g
server-1.properties & server-2.properties

You need to then specify different broker.id & port in each of those files to make them unique ..

   config/server-1.properties:
     broker.id=1
     port=9093
     log.dir=/tmp/kafka-logs-1

   config/server-2.properties:
     broker.id=2
     port=9094
     log.dir=/tmp/kafka-logs-2

And then start multiple instances with following commands

> bin/kafka-server-start.sh config/server-1.properties &
> bin/kafka-server-start.sh config/server-2.properties &

For more details check Step 6: Setting up a multi-broker cluster

user2720864
  • 8,015
  • 5
  • 48
  • 60
0

I faced the same issue while setting up multiple brokers.

The step I was missing was:

While editing the config file: config/server-1.properties: broker.id=1 listeners=PLAINTEXT://:9093 log.dir=/tmp/kafka-logs-1

need to update the LOG BASICS section as well(see below):

####################### Log Basics

A comma separated list of directories under which to store log files

log.dirs=/tmp/kafka-logs-1
Ananth Joshi
  • 11
  • 2