0

My source cluster has a set of topics to which data is being published. I have these topic names specified in the mirrormaker properties file. I have not created the topics in the target cluster. My assumption is that MM 2.0 will automatically create these topics in the target cluster if they don't exist. But when I try to consume from the topic in the target cluster, I see the following message -

[2020-06-11 21:45:19,301] WARN [Consumer clientId=consumer-console-consumer-78249-1, groupId=console-consumer-78249] Error while fetching metadata with correlation id 4790 : {topic1=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)

Does it mean the topics and partitions need to be created beforehand in the target cluster?

bdev03
  • 375
  • 1
  • 4
  • 19

3 Answers3

1

No, MM 2.0 takes responsibility to create a replicated topic in the target cluster. The topic name in the target cluster will be sourceAlias..

Refer this for more details: https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP382:MirrorMaker2.0-Config,ACLSync

PAVAN PURANIK
  • 41
  • 3
0

I was looking at the topic with the same name as in the source topic but the topic name in the target cluster is .topic1.

bdev03
  • 375
  • 1
  • 4
  • 19
0

It depends on your kafka broker configuration. Make sure kafka broker property auto.create.topics.enable should be true. If it is false, MM2 will not be able create topic or partitions automatically

How to create topics in apache kafka?

krishna k maurya
  • 47
  • 9