Stack Overflow
Stack Overflow

Questions tagged [spring-kafka]

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions.

The Spring for Apache Kafka (spring-kafka) project applies core Spring concepts to the development of Apache Kafka-based messaging solutions. It provides a "template" as a high-level abstraction for sending messages. It also provides support for Message-driven POJOs with @KafkaListener annotations and a "listener container". These libraries promote the use of dependency injection and declarative. In all of these cases, you will see similarities to the JMS support in the Spring Framework and RabbitMQ support in Spring AMQP.

has a module spring-integration-kafka providing channel adapters and gateways based on spring-kafka.

Resources

See Also

4185 questions
74
votes
5 answers

Synchronising transactions between database and Kafka producer

We have a micro-services architecture, with Kafka used as the communication mechanism between the services. Some of the services have their own databases. Say the user makes a call to Service A, which should result in a record (or set of records)…
Yoni Gibbs
  • 6,518
  • 2
  • 24
  • 37
60
votes
2 answers

Spring-Kafka vs. Spring-Cloud-Stream (Kafka)

Using Kafka as a messaging system in a microservice architecture what are the benefits of using spring-kafka vs. spring-cloud-stream + spring-cloud-starter-stream-kafka ? The spring cloud stream framework supports more messaging systems and has…
Eike Behrends
  • 855
  • 2
  • 7
  • 11
52
votes
4 answers

Simple embedded Kafka test example with spring boot

Edit FYI: working gitHub example I was searching the internet and couldn't find a working and simple example of an embedded Kafka test. My setup is: Spring boot Multiple @KafkaListener with different topics in one class Embedded Kafka for test…
Yuna Braska
  • 721
  • 1
  • 7
  • 15
45
votes
11 answers

Spring Kafka The class is not in the trusted packages

In my Spring Boot/Kafka application before the library update, I used the following class org.telegram.telegrambots.api.objects.Update in order to post messages to the Kafka topic. Right now I use the following…
alexanoid
  • 24,051
  • 54
  • 210
  • 410
40
votes
5 answers

How to acknowledge current offset in spring kafka for manual commit

I am using Spring Kafka first time and I am not able to use Acknowledgement.acknowledge() method for manual commit in my consumer code. please let me know if anything missing in my consumer configuration or listener code. or else is there other way…
user2550140
  • 401
  • 1
  • 4
  • 3
32
votes
4 answers

Kafka producer TimeoutException: Expiring 1 record(s)

I am using Kafka with Spring-boot: Kafka Producer class: @Service public class MyKafkaProducer { @Autowired private KafkaTemplate kafkaTemplate; private static Logger LOGGER =…
Prakash P
  • 3,582
  • 4
  • 38
  • 66
28
votes
6 answers

spring kafka No type information in headers and no default type provided

I've got a spring boot app that defines: a REST Controller that writes to a kafka topic, STREAM_TOPIC_IN_QQQ a KafkaListener that reads from STREAM_TOPIC_IN_QQQ (groupId="bar") and logs a KStream that peeks the topic and logs it, converts it to…
mconner
  • 1,174
  • 3
  • 12
  • 24
27
votes
2 answers

In Spring Kafka, do I need to add the @EnableKafka annotation to my application?

I see some people adding @EnableKafka to their spring boot application and I was wondering why. I have a working spring boot kafka producer and consumer and I didn't use @EnableKafka. So, why do people need to add it explicitly? Thank you.
crillin
  • 273
  • 1
  • 3
  • 5
27
votes
2 answers

Testing a @KafkaListener using Spring Embedded Kafka

I am trying to write a unit test for a Kafka listener that I am developing using Spring Boot 2.x. Being a unit test, I don't want to start up a full Kafka server an instance of Zookeeper. So, I decided to use Spring Embedded Kafka. The definition of…
riccardo.cardin
  • 7,971
  • 5
  • 57
  • 106
24
votes
2 answers

How to write Kafka consumers - single threaded vs multi threaded

I have written a single Kafka consumer (using Spring Kafka), that reads from a single topic and is a part of a consumer group. Once a message is consumed, it will perform all downstream operations and move on to the next message offset. I have…
user3842182
  • 371
  • 2
  • 5
  • 8
23
votes
2 answers

When to use ConcurrentKafkaListenerContainerFactory?

I am new to kafka and i went through the documentation but I couldn't understand anything. Can someone please explain when to use the ConcurrentKafkaListenerContainerFactory class? I have used the Kafkaconsumer class but I see…
Rahul Gupta
  • 1,079
  • 2
  • 15
  • 27
23
votes
1 answer

Dead letter queue (DLQ) for Kafka with spring-kafka

What is the best way to implement Dead letter queue (DLQ) concept in Spring Boot 2.0 application using spring-kafka 2.1.x to have all messages that were failed to be processed by @KafkaListener method of some bean sent to some predefined Kafka DLQ…
Eugene Khyst
  • 9,236
  • 7
  • 38
  • 65
21
votes
2 answers

How to get message by key from kafka topic

I try to get message by key from kafka. I found the only one solution is to use StateStore but I think it might be not a good idea. How to get message by key from kafka topic? Is it a good idea to use StateStore for this operation?
Іван Гладуш
  • 375
  • 1
  • 2
  • 8
21
votes
5 answers

org.apache.kafka.common.KafkaException: Failed to construct kafka consumer

I am manually starting Zookeeper, then Kafka server and finally the Kafka-Rest server with their respective properties file. Next, I am deploying my Spring Boot application on tomcat In the Tomcat log trace, I am getting the Error…
somnathchakrabarti
  • 3,026
  • 10
  • 69
  • 92
20
votes
2 answers

Apache Kafka: Replay messages in a topic

I'm considering using Apache Kafka as an event store for storing events within a microservice. One thing that I read through various blogs is that Kafka can be considered to be a single source of truth, where Kafka log will store all the events for…
rm12345
  • 1,089
  • 3
  • 18
  • 32
1
2 3
…
99 100