How to list Kafka topics while I am using strimzi operator

Question

Hey I am usin Kafka Strimzi. I created my kafkaTopic and KafkaUser using following yml files:

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaUser
metadata:
  name: my-user
  labels:
    strimzi.io/cluster: my-cluster
spec:
  authentication:
    type: tls
  authorization:
    type: simple
    acls:
      # Example consumer Acls for topic my-topic using consumer group my-group
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Read
        host: "*"
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Describe
        host: "*"
      - resource:
          type: group
          name: my-group
          patternType: literal
        operation: Read
        host: "*"
      # Example Producer Acls for topic my-topic
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Write
        host: "*"
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Create
        host: "*"
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: Describe
        host: "*"

and

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaTopic
metadata:
  name: my-topic
  labels:
    strimzi.io/cluster: my-cluster
spec:
  partitions: 1
  replicas: 1
  config:
    retention.ms: 7200000
    segment.bytes: 1073741824

[kafka@my-cluster-zookeeper-0 kafka]$ bin/kafka-topics.sh --list --zookeeper 10.101.97.123:2181

While I am using this command I m getting this error don't know how to fix this.

Exception in thread "main" kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
    at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:262)
    at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:258)
    at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:119)
    at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1865)
    at kafka.admin.TopicCommand$ZookeeperTopicService$.apply(TopicCommand.scala:360)
    at kafka.admin.TopicCommand$.main(TopicCommand.scala:55)
    at kafka.admin.TopicCommand.main(TopicCommand.scala)

having this issue while listing my kafkaTopics .Please help me.

score 8 · Accepted Answer · answered Nov 17 '20 at 11:43

8

You cannot do that because Zookeeper connection is encrypted but you can go in this way for example:

kubectl exec -it my-cluster-kafka-0 -c kafka -- bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list

Use the Kafka servers bootstrap connection and not the one to ZooKeeper.

Of course, it's the way for checking right on Kafka but you can also check the corresponding created KafkaTopic resources with:

kubectl get kafkatopic

answered Nov 17 '20 at 11:43

ppatierno

9,431
1
30
45

kubectl exec -it my-cluster-kafka-0 -c kafka -- bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list ....from where shoud i use this command ...form the kafka pod? – rohit290554 Nov 17 '20 at 12:19
from your local laptop, it's using the `kubectl exec` for running a bash command on one of the broker pod my-cluster-kafka-0. Or you can ssh into one of the pod and running just the last part `bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list ` – ppatierno Nov 17 '20 at 12:22
Thanks I got my result when I used the second command from the pod – rohit290554 Nov 17 '20 at 12:25
I think the suggestion to run it from the Kafka pod is not ideal since you would create another JVM in there etc. Better approach is to run separate pod ... e.g.: `kubectl -n kafka run kafka-topics -ti --image=strimzi/kafka:0.20.0-kafka-2.6.0 --rm=true --restart=Never -- bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list`. – Jakub Nov 18 '20 at 00:27
Updated to current versions: kubectl -n yourNamespace run kafka-topics -ti --image=quay.io/strimzi/kafka:0.28.0-kafka-3.1.0 --rm=true --restart=Never -- bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list – Nimrod Yonatan Ben-Nes May 09 '22 at 15:21

score 7 · Answer 2 · answered Sep 28 '21 at 01:54

7

Provided all your topics are created via CRD, the fastest way is

kubectl get kt -n <your-namespace>

You can see more about it here

answered Sep 28 '21 at 01:54

Tung

163
1
4
7

How to list Kafka topics while I am using strimzi operator

2 Answers2