Stack Overflow
Stack Overflow

Questions tagged [node-kafka-streams]

Node-Kafka-Streams is a nodejs implementation from nodefluent of Apache Kafka's stream processing library Kafka Streams.

Apache Kafka is distributed stream processing platform, and offers a Java-written stream processing library called Kafka Streams (aka Streams API).

Node-Kafka-Streams is a reimplementation of Kafka Streams in nodejs.

11 questions
4
votes
1 answer

Kafka Streams - Multiple Threads or Multiple Instances with same application id

I have a Kafka Streams application that consumes from topic 'A' with 10 partitions with around 10k messages per second. I am confused about what will be better for my application. To run multiple Kafka Streams application instances with same…
Nandish Kotadia
  • 441
  • 1
  • 6
  • 21
3
votes
0 answers

Error connecting to producer : Error: ssl.certificate.location failed

Environment Information: OS [e.g. Mac, Arch, Windows 10]: Windows10 Node Version [e.g. 8.2.1]:10.4.1 NPM Version [e.g. 5.4.2]:6.4.1 node-rdkafka version [e.g. 2.3.3]:2.5.1 Steps to Reproduce node-rdkafka Configuration Settings: var producer = new…
Mrinal Deo
  • 81
  • 2
  • 10
2
votes
0 answers

how to resolve refreshbrokermetadata failed issue in node?

Getting below issue while integrating kafka with node after certain time of application get started in node server. There is not any log in kafka server as well as it is up and running. /app/node_modules/async/dist/async.js:5329 return…
pprajapati
  • 41
  • 4
1
vote
0 answers

NodeJs - Kafka streams does not output messages to console

I am using node-fluent/kafka-streams to connect to a Kafka cluster launched using Strimzi. The quick start example does not emit any message - it only shows the message stream started, as kafka consumer is ready.. I can confirm that messages are…
Chain Head
  • 13
  • 3
1
vote
1 answer

parallel writes different topics from single stream topic

I have a stream which gives messages map to two different map() call and further is filtered and written to two different topics. KStream[] stream = builder.stream("source-topic"); stream.map(logic1OnData).filter( …
Sudarshan sridhar
  • 201
  • 5
  • 18
0
votes
1 answer

sending http/2 POST request in Node.js

How can send an http/2 post request in Node.js? I want to send the following request. curl --http2 "POST" "http://hostname:8088/query-stream" -d $'{"sql": "SELECT * FROM `USERPROFILE` EMIT CHANGES;", "properties": {"ksql.streams.auto.offset.reset":…
Armen Chakhalyan
  • 349
  • 1
  • 2
  • 4
0
votes
1 answer

Can kafka-streams listen to two topics and wait for the other event that matches the id and once it receives post it to another topic?

I have a two different kafka topics which has same field Id in it. Can a stream listen to these two topics and write a new message to the third topic when it receives same id on both those topics?
SectumSempra
  • 11
  • 1
  • 3
0
votes
0 answers

Kafka Streams - Can I use same state dir for multiple streams instances with same application id

I have multiple Kafka Streams instances with same application.id which runs on the same machine. So can I use same state.dir for each instance or a different state.dir I read it somewhere that it can lock global state directory if I use same…
Nandish Kotadia
  • 441
  • 1
  • 6
  • 21
0
votes
2 answers

Unable to install npm package (kafka-streams)

I am trying to use npm package kafka-streams but getting below error: PS D:\Projects\POCs\kstreams-poc> npm install kafka-streams > node-rdkafka@2.7.1 install D:\Projects\POCs\kstreams-poc\node_modules\node-rdkafka > node-gyp…
Aftab
  • 2,863
  • 32
  • 41
0
votes
1 answer

What is causing this intermittent issue with Node Kafka Streams?

I have a Kafka producer and consumer. The producer does this: const returnMessage = { prop1: 'some string', prop2: 'another string', prop3: nestedObject }; console.log(JSON.stringify(returnMessage)) await…
Boris K
  • 3,442
  • 9
  • 48
  • 87
0
votes
0 answers

Node Kafka Streams hangs after first connection

I have an HTTP2 application which creates a new Kafka streams producer and sends messages to it on receiving a message from an incoming stream. The first time it connects and functions properly. On subsequent streams, it fails without throwing any…
Boris K
  • 3,442
  • 9
  • 48
  • 87