8

New to Kafka.

Having a difficult time figuring out how to use the java-api to create a topic in in the 0.9 release.

This answer deals with 0.8.

I see that the Cluster has methods for querying the partition info, topic, etc. But I can't see how I can dynamically create a topic.

Community
  • 1
  • 1
hba
  • 7,406
  • 10
  • 63
  • 105

1 Answers1

11

For Kafka, the cluster determines how/if you're able to create topics. If you want to be able to create topics on the fly, the easiest way is to use auto.create.topics.enable on your cluster. Then when you send a message to a topic that doesn't exist, the cluster creates the topic with the cluster default partitions and replication factor. If you don't have/want this feature enabled, there are not methods that I know of in the Kafka clients library.

If you're determined, you can look into the internals of the kafka-topics.sh, which is where you'll find how Kafka creates topics with the core api.

Edit
Now Kafka offers Admin API which allows you to programmatically create topics (among other things). See official API docs. (kafka version 1.0.0)

kubacech
  • 117
  • 3
  • 11
David Krieg
  • 310
  • 2
  • 6
  • This doesn't answer his question at all. He is also asking on creating topics using the java api, the link you gave is written in scala. – Chee Loong Soon Apr 01 '16 at 19:39
  • 6
    This actually does answer the question though, there is not a Java client API for this. There's Scala admin API that does this, but there's no Java API in the client to programmatically create topics, because it's only available as an admin feature/something provided at the cluster level. @GwenShapira backed me up on this, so I'd invite you to read more about Kafka. The answer is either: A. You don't, you enable the cluster to do it automatically on new messages to a topic. Or B. You have to dive into the Scala only admin API. PS, Scala classes are usable from Java. – David Krieg May 02 '16 at 23:18