How can I retrieve Kafka messages inside a controller in Spring Boot?

Question

The messages created by the producer are all being consumed as expected.
The thing is, I need to create an endpoint to retrieve the latest messages from the consumer.
Is there a way to do it? Like an on-demand consumer?

I found this SO post but is only to consume the last N records. I want to consume the latest without caring about the offsets.

Spring Kafka Consumer, rewind consumer offset to go back 'n' records

I'm working with Kotlin but if you have the answer in Java I don't mind either.

Consumers should be long-running. What do you mean "on demand"? If you don't care about offsets, just use `consumer.seekToEnd` — OneCricketeer, Jan 24 '23 at 16:45
"On demand" means "on demand", as in, whenever I call certain function, etc. But don't worry, ChatGPT already gave me the solution. — Alberto Alegria, Jan 25 '23 at 09:20

score 1 · Accepted Answer · answered Jan 25 '23 at 13:52

There are several ways to create listener containers dynamically; you can then start/stop them on demand. To get the records back into the controller, you'd need to use something like a blocking queue, or make the controller itself a MessageListener.

These answers show a couple of techniques for creating containers on demand:

How to dynamically create multiple consumers in Spring Kafka

Kafka Consumer in spring can I re-assign partitions programmatically?

Thanks for your answer! Dynamic consumers is what actually helped me — Alberto Alegria, Jan 26 '23 at 14:13

How can I retrieve Kafka messages inside a controller in Spring Boot?

1 Answers1