50

I want to see the retention period set for a particular topic. Is there any command? I tried with

bin/kafka-topics.sh --zookeeper hostname:2181 --alter --config retention.ms=172800000 --topic <topic_name>

for deleting 2 days old data. But I want to see how many days are set in the retention period for all the topics.

Naman
  • 27,789
  • 26
  • 218
  • 353
manoj
  • 601
  • 1
  • 5
  • 3

5 Answers5

72

If you have altered a topic and want to view the topic configuration the following command will be helpful

kafka-topics.sh --zookeeper localhost:2181 --describe --topics-with-overrides

This will describe only the topics along with configurations that have configurations set that differ from the cluster defaults.

If you want to view the configurations for all topic Either you can view these properties log.retention.hours or log.retention.ms in server.properties in kafka config directory.

Naman
  • 27,789
  • 26
  • 218
  • 353
Abhimanyu
  • 2,710
  • 2
  • 25
  • 42
15

See config 'log.retention.hours' in Kafka Doc:

The number of hours to keep a log file before deleting it (in hours), tertiary to log.retention.ms property

Default value is 168 hours which is 7 days.

amethystic
  • 6,821
  • 23
  • 25
4

Alternative ways:

  1. use call sh-command through a container
docker run --rm -it confluentinc/cp-kafka:latest sh -c "kafka-topics --zookeeper 11.22.33.44:5555 --describe --topic topic-name"

enter image description here

  1. use Kafka Tool

enter image description here

Take into account that it would be displayed just params different from default values.

At the example above, the retention.ms is 14 days, when default one is 7 days.

vladimir
  • 13,428
  • 2
  • 44
  • 70
2

Simply the below command will help

kafka-topics.sh --bootstrap-server server_ip:9092 --describe --topic topic_name

Along with other infos , it will print delete.retention.ms.

A sample output will be

Topic:TOPIC NAME    PartitionCount:6    ReplicationFactor:1 Configs:compression.type=gzip,segment.bytes=1073741824,retention.ms=100,max.message.bytes=100001200,delete.retention.ms=100000

Topic: TOPIC NAME   Partition: 0    Leader: 2   Replicas: 2 Isr: 2
Topic: TOPIC NAME   Partition: 1    Leader: 3   Replicas: 3 Isr: 3
Topic: TOPIC NAME   Partition: 2    Leader: 1   Replicas: 1 Isr: 1
Topic: TOPIC NAME   Partition: 3    Leader: 2   Replicas: 2 Isr: 2
Topic: TOPIC NAME   Partition: 4    Leader: 3   Replicas: 3 Isr: 3
Topic: TOPIC NAME   Partition: 5    Leader: 1   Replicas: 1 Isr: 1
Mikhail2048
  • 1,715
  • 1
  • 9
  • 26
Pramod Kishore
  • 307
  • 2
  • 8
0

If you want to get the retention.ms or retention.bytes or any other config for any topic, you can simply use this command:

kafka-configs.sh --bootstrap-server <BOOTSTRAP_SERVER> --describe --topic <TOPIC_NAME> --all

Make sure to put here the address of the broker and the desired topic. This will output all configs defined for a particular topic, including the retention. Hope it helped :)

Mikhail2048
  • 1,715
  • 1
  • 9
  • 26