How can I delete a topic in Apache Kafka?

Question

I need to delete a topic in Kafka 0.8.2.2.3. I have used the below command for deleting the topic:

bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic DummyTopic

The command executed successfully, but when I run a command to list the topics, I could see that the topic is still there and it shows marked for deletion.

bin/kafka-topics.sh --list --zookeeper localhost:2181
DummyTopic - marked for deletion

And when I create the topic DummyTopic, it outputs the exception. The topic already exists, and below is the stack trace:

Error while executing topic command Topic "DummyTopic" already exists.
kafka.common.TopicExistsException: Topic "DummyTopic" already exists.
    at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:248)
    at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:233)
    at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:92)
    at kafka.admin.TopicCommand$.main(TopicCommand.scala:54)
    at kafka.admin.TopicCommand.main(TopicCommand.scala)

How can I delete this topic?

Apache Kafka never deletes a topic marked for deletion if that topic has producers still producing to it, or consumers still consuming from it, or messages left hanging out in the queue. One way to try to try to force it is to restart Kafka. Or if that doesn't work, go in under the hood and Delete the directory that is your topic name under /var/local/kafka/data and then restart Kafka then re-issue the delete command. I wish Apache Kafka had a "nuke the bleeping topic" option so the developer can issue the command: "Really, totes for real this time nuke the bleeping topic no sass please". — Eric Leschinski, Sep 07 '16 at 21:07
bin/kafka-topics.sh –delete –zookeeper localhost:2181 –topic — Avinav Mishra, Feb 21 '17 at 02:16
Option zookeeper is deprecated, use --bootstrap-server instead. https://stackoverflow.com/a/53429129/5456789 — Amir Azizkhani, Jul 06 '22 at 10:40
This worked for me (within docker): bin/kafka-topics --delete --topic DummyTopic --bootstrap-server localhost:29092 — Enrico Giurin, Nov 28 '22 at 03:46

score 139 · Accepted Answer · edited Jul 23 '19 at 20:13

139

Deletion of a topic has been supported since 0.8.2.x version. You have to enable topic deletion (setting delete.topic.enable to true) on all brokers first.

Note: Ever since 1.0.x, the functionality being stable, delete.topic.enable is by default true.

Follow this step by step process for manual deletion of topics

Stop Kafka server
Delete the topic directory, on each broker (as defined in the logs.dirs and log.dir properties) with rm -rf command
Connect to Zookeeper instance: zookeeper-shell.sh host:port
From within the Zookeeper instance:
1. List the topics using: ls /brokers/topics
2. Remove the topic folder from ZooKeeper using: rmr /brokers/topics/yourtopic
3. Exit the Zookeeper instance (Ctrl+C)
Restart Kafka server
Confirm if it was deleted or not by using this command kafka-topics.sh --list --zookeeper host:port

edited Jul 23 '19 at 20:13

OneCricketeer

179,855
19
132
245

answered Nov 05 '15 at 06:52

Ravindra babu

37,698
11
250
211

ok...thanks...I will try restarting the kafka cluster..:) – Rishi Arora Nov 05 '15 at 08:21
3

Link is broken! – Rubbal Feb 01 '17 at 09:39
10

Where is the "/brokers/topics" directory? cant seem to find that folder – Vipresh Jul 20 '17 at 11:41
1

Use `zookeeper-shell.sh` to get to zookeeper and `ls /brokers/topics` – f01 Oct 05 '17 at 02:31
If I check the docs on bin/kafka-topics.sh I see a --delete option. But using it does not work. Is it still in beta? – IcedDante Feb 03 '18 at 01:32
1

NOTE: If u use --delete from kafka-topics.sh ---> make sure there is no CONSUMER running ---> and you set 'delete.topic.enable:true' in server.properties file on all kafka brokers. – hariszhr Feb 08 '18 at 18:40
1

`kafka.admin.TopicCommand` shows how to do this programmatically. – F. P. Freely Jun 19 '18 at 17:45
1

how to delete all topics not just one? – Aubergine Aug 31 '18 at 11:31
144

i cant believe its so difficult to delete a topic – Erik K Aug 16 '19 at 20:50
@ErikK Is there a better way? – No Holidays Jun 20 '20 at 09:26
7

`rmr` command is deprecated. you can use `deleteall` instead of `rmr` – Nafaz M N M Nov 02 '20 at 16:35
2

I wouldn't recommend this rm -rf approach for prod clusters. The proper way is to enable delete topic in server.properties. Use --delete Wait for the offsets to expire naturally on these topics. The zookeeper shell uninstall works too but causes controller node to change. – c0der512 Feb 01 '21 at 19:02
windows users can run zookeeper-shell.bat , for example: zookeeper-shell.bat localhost:2821. btw, there is deleteall command – AsfK Apr 07 '21 at 13:35
6

I visit this answer every couple of weeks. Sometimes multiple times in a week. – Neelesh Jun 23 '21 at 07:04
Don't delete the folders while kafka is still running! (an easy concept to grasp which I am learning currently the hard way how to deal with) – Zap Oct 06 '21 at 16:12
3

Hmm.. now we just need to use the command mentioned in the query `bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic DummyTopic` and voila it's done. – Yash Jul 14 '22 at 22:05

How can I delete a topic in Apache Kafka?

1 Answers1

Linked