1

Is it possible to remove data from Kafka after that data consumed?

I am using Kafka and Zookeeper combined to exchange some data between 2 nodes. I am having trouble with memory of Kafka because I am inserting so many data. In this way I need to clean datas from Kafka after I consumed them. Is it possible?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Bu_A
  • 41
  • 1
  • 5
  • 1
    That is not possible, check SO post: https://stackoverflow.com/questions/28586008/delete-message-after-consuming-it-in-kafka, also hortonworks: https://community.hortonworks.com/questions/50962/delete-message-after-consuming-it-from-kafka-topic.html – dkb Oct 05 '18 at 11:19
  • What is the use case ? There is one using admin client - https://cwiki.apache.org/confluence/display/KAFKA/KIP-204+%3A+Adding+records+deletion+operation+to+the+new+Admin+Client+API but that will not be useful for deleting record after every read. – Ajay Srivastava Oct 05 '18 at 11:25
  • The Kafka heap space isn't affected by deleting messages – OneCricketeer Oct 05 '18 at 14:19

1 Answers1

0

You can try out changing the retention time.

kafka-topics.sh --zookeeper localhost:13003 --alter --topic MyTopic --config retention.ms=1000

But be sure, as the message will be deleted irrespective of weather it is consumed or not.

Aman Saurav
  • 751
  • 9
  • 28