How to delete multiple topics in Apache Kafka

Question

Assuming that I have a number of topics with the same prefix, e.g:

giorgos-topic1
giorgos-topic2
giorgos-topic3
...

The command used for deleting a single topic (say giorgos-topic1) is the following:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic giorgos-topic1

Is it possible to delete multiple topics using a single command and possibly a regular expression/wildcard (e.g. giorgos-*) instead of typing all the topic names that need to be deleted one by one?

Does anyone what what should be the regex for digits only topic-names? — Guy_g23, May 08 '23 at 07:31

score 83 · Accepted Answer · edited Jun 28 '19 at 11:56

83

Yes you can use regex-like expressions when deleting topics with the kafka-topics.sh tool:

For example, to delete all topics starting with giorgos-:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic 'giorgos-.*'

Using the Admin APIs, you can also delete several topics at once, see AdminClient.deleteTopics

edited Jun 28 '19 at 11:56

Alexander Popov

23,073
19
91
130

answered Feb 05 '18 at 10:45

Mickael Maison

25,067
7
71
68

1

how about deleing multiple patterns, e.g., foo.* bar.* ? – petertc Oct 16 '18 at 03:02
It seems AdminClient.deleteTopics supports topic names instead of topic name patterns in its arguments. Is there a way to delete by patterns without querying in advance? – David Jun 17 '22 at 07:38

score 8 · Answer 2 · answered Nov 13 '18 at 12:31

8

In cases where regex is not possible we can use a comma seperated list of topic names for the deletion of topics.

kafka-topics.sh --zookeeper 10.0.0.160:2181 --delete --topic giorgos-topic1,giorgos-topic2,giorgos-topic3,...

answered Nov 13 '18 at 12:31

Aman Saurav

751
9
28

1

Regex is always possible. For this example, `giorgos-topic[123]`. You can use OR too. `(some-topic|my-other-topic_1)` – OneCricketeer Jan 18 '20 at 02:26

score 4 · Answer 3 · answered Mar 24 '21 at 01:17

4

Just to add to the accepted answer, the * wildcard needs to be preceded by '.'

In my experience:

--delete --topic '_confluent-controlcenter-5-3-1-1-.*' works

--delete --topic '_confluent-controlcenter-5-3-1-1-*' DOESNT work

Note: I would've added this as a comment but don't have enough rep.

answered Mar 24 '21 at 01:17

famen

55
1
5

That's because it's a regular expression, not glob syntax. Also, the accepted answer did imply you needed a `.*` – Cardin Apr 30 '21 at 09:30

score 3 · Answer 4 · answered Jan 07 '19 at 04:13

3

Please add single quote ('giorgos-.* ') if it complains like no matches found: giorgos-.*

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic 'giorgos-.*'

answered Jan 07 '19 at 04:13

kundan bora

3,821
2
20
29

Giorgos Myrianthous · Answer 5 · 2022-09-19T11:22:48.507

2

Recent Kafka versions will remove the Zookeeper dependency and going forward you need to reference the brokers, through --boostrap-server:

kafka-topics \
    --bootstrap-server localhost:9092,localhost:9093,localhost:9094 \
    --delete \
    --topic 'giorgos-.*'

edited Sep 19 '22 at 11:22

answered Mar 14 '22 at 16:54

Giorgos Myrianthous

36,235
20
134
156

How to delete multiple topics in Apache Kafka

5 Answers5

Linked