I want to implement request-response pattern with kafka stream,i use spring boot kafka,which add some data as header,named correlation id,but when the request message processed by kafka stream API,header data would be lost,can't send to response topic! How can i solve it,or use another method??
Asked
Active
Viewed 946 times
0
-
How is the header data "lost"? Kafka itself does not just lose parts of the message payload. Where did you (or Spring Boot) put that header? – Thilo Apr 19 '20 at 05:22
-
no,some re-partition operation in kafka stream,then header lost – zydzjy Apr 19 '20 at 05:30
-
@zydzjy Why don't you try putting header in kafka message payload? And what kind of oeprator you used which cause re-partition? – Tuyen Luong Apr 19 '20 at 09:34
-
@TuyenLuong I use spring boot kafkaTemplate,which use correlationId as message header,and processors include re-partition operations – zydzjy Apr 19 '20 at 11:43
-
`no,some re-partition operation in kafka stream,then header lost` -- what version are you using? Kafka Streams supports headers as of `v2.0` (cf https://stackoverflow.com/questions/46736484/is-it-possible-to-access-message-headers-with-kafka-streams/46738514) -- but I am not really sure what you want to achieve. Kafka Streams is build for stream processing, not req/resp: of you you meant that your requests are messages in an input topic and you produce a response to an output topic (this should work, but I would not call it a req/resp pattern; that is really stream processing). – Matthias J. Sax Apr 20 '20 at 00:12
1 Answers
1
You wouldn't have a response for a request other than (201, accepted)
data would be lost
No, it wouldn't
can't send to response topic!
Why not?
How can i solve it,or use another method??
Use WebSockets to get your response.
And you might want to watch this talk that uses multiple Spring apps.
https://www.confluent.io/kafka-summit-lon19/hello-streams-introducing-the-first-streams-mindset/
OneCricketeer
- 179,855
- 19
- 132
- 245