How to purge or delete a topic in kafka 2.1.0 version

Question

Would like to share different ways to purge or delete a kafka topic in 2.1.0 version. I've found similar question here Purge Kafka Topic however, the accepted answer has been deprecated and it works on Kafka version 0.8 and below hence, creating this question with answer.

This is not a duplicate question.

Possible duplicate of [Purge Kafka Topic](https://stackoverflow.com/questions/16284399/purge-kafka-topic) — OneCricketeer, Jan 27 '19 at 18:37
Just because nothing was accepted doesn't mean you need to duplicate the answers there — OneCricketeer, Jan 27 '19 at 18:37

score 7 · Accepted Answer · answered Jan 27 '19 at 00:17

Kafka by default keeps the messages for 168 hrs which is 7 days. If you wanted to force kafka to purge the topic, you can do this by multiple ways. Let’s see each in detail.

1. Using kafka-configs.sh command

Temporarily change the retention policy to 1 sec.

kafka-configs.sh --zookeeper localhost:2181 --alter --entity-type topics --add-config retention.ms=1000 --entity-name text_topic

You can check the current value of retention policy by running below command.

kafka-configs.sh --zookeeper localhost:2181 --entity-type topics --describe --entity-name text_topic
Configs for topic 'text_topic' are retention.ms=1000

Wait for 1 sec and remove the retention policy config which will set it back to default.

kafka-configs.sh --zookeeper localhost:2181 --alter --entity-type topics --delete-config retention.ms --entity-name text_topic

2. Delete topic and re-create

Before we delete an existing topic, first get the partition and replica of the current topic as you would need these to re-create the topic. You can get this information by running describe of the topic

kafka-topics.sh --zookeeper localhost:2181 --describe --topic text_topic

Topic:text_topic        PartitionCount:3        ReplicationFactor:3     Configs:
        Topic: text_topic       Partition: 0    Leader: 0       Replicas: 0     Isr: 0

Delete the topic.

kafka-topics.sh --zookeeper localhost:2181 --delete --topic text_topic

Re-create the topic with replication and partition details.

kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 3 --topic text_topic

3. Manually delete the data from kafka logs.

Stop zookeeper and kafka from all nodes.
Clean kafka logs from all nodes. kafka stores its log files at /tmp/kafka-logs/MyTopic-0 where /tmp/kafka-logs is specified by the log.dirattribute
Restart zookeeper and kafka.

Hope this helps !!

How to purge or delete a topic in kafka 2.1.0 version

1 Answers1