Stack Overflow
Stack Overflow

Questions tagged [kraft]

KRaft is an Apache Kafka runtime mode that uses the Raft protocol internally between Controller and Broker servers rather than relying on an external Apache Zookeeper installation.

37 questions
9
votes
1 answer

Kafka 3.3 with Kraft - Controller ID keeps changing every second

I have a three node Kafka cluster with Kraft and without Zookeeper. The cluster Controller ID changes almost every second. Is this normal? As I understand, it should change only when some fault in the cluster forces re-election. Kafka Version: 3.3
Dojo
  • 5,374
  • 4
  • 49
  • 79
5
votes
2 answers

Apache Kafka in kraft mode fails frequently

We have created a 3 node kafka-3.3.1 cluster in kraft mode. This is based on bitnami-kafka image. Basic configuration for all nodes are (port number is different for each and other changes as required) KAFKA_ENABLE_KRAFT: 'yes' …
donnie
  • 2,981
  • 2
  • 16
  • 24
4
votes
1 answer

Getting error while starting Kafka(3.4.0) with KRaft on windows machine

I am trying to start the Kafka with KRaft by following doc here - https://kafka.apache.org/quickstart on windows machine. Note - The same version of kafka works perfectly fine with zookeeper Step 1 - generate uuid (.\bin\windows\kafka-storage.bat…
Arvind Kumar
  • 459
  • 5
  • 21
3
votes
1 answer

KAFKA_CFG_LISTENERS must include a listener for CONTROLLER

Edit: For anyone reading this (question maintained below) here is some short summary which might resolve your issue: The Bitnami Kafka 3.4 image appears to have been updated recently to use Kraft instead of Zookeeper by default To revert back to…
FreelanceConsultant
  • 13,167
  • 27
  • 115
  • 225
3
votes
1 answer

Kafka cluster using KRaft, mTLS and StandardAuthorizer not starting up, getting AuthorizerNotReadyException

I'm trying to create a new Kafka cluster using the new KRaft mode and using SSL certificates for authentication and authorization (along with the new StandardAuthorizer). I'm using the Bitnami Kafka image, version 3.3.1. This is the relevant part of…
jdias
  • 45
  • 6
2
votes
1 answer

Bitnami Kafka KRaft Failed to configure SaslClientAuthenticator

I'm trying to migrate Kafka with ZooKeeper to KRaft and I have a problem with inter-broker secured setup. Everithing works nicely with disabled authorizer via CONTROLLER with PLAINTEXT, but when I'm trying to setup it, Kafka throws errors below.…
Nik_Avrinsky
  • 21
  • 1
2
votes
1 answer

Need Kafka KRAFT SASL_PLAINTEXT

I’m trying to get Kafka in kraft mode up n’ running with SASL_PLAINTEXT I’ve got a functioning kafka broker/controller up n’ running locally, without SASL using this…
Firecow
  • 501
  • 1
  • 5
  • 11
2
votes
2 answers

Apache Kafka KRaft mode topology best practices

My question concerns the recommended topology of Kafka brokers and controllers in KRaft mode. Now, according to best practices with zookeeper we are supposed to create: {3,5,7} Zookeeper nodes {3,5,7} Three Kafka broker nodes This is a well known…
Aladin
  • 492
  • 1
  • 8
  • 21
2
votes
2 answers

Kafka: Find Controller ID in a cluster using Kraft protocol

How to find the current controller ID, preferably using command-line, on a Kafka cluster which is using Kraft. Kafka Version: 3.3
Dojo
  • 5,374
  • 4
  • 49
  • 79
1
vote
0 answers

Frequent connection timeout in the kafkastream

We have multiple kafka clusters dedicated to our data pipeline. One of our production clusters was setup a year back. The data path is something like source topic -> kafkastreams -> destination topics -> kafkaconnect We started getting problems…
donnie
  • 2,981
  • 2
  • 16
  • 24
1
vote
0 answers

kafka kraft controller replacement

I want to replace controller for my kafka kraft cluster (without downtime if it possible). All my actions. All my actions lead to the fact that the cluster is not recoverable. My actions: change quorum from 3 to 2 on each brokers and…
Андрей Леушкин
  • 41
  • 1
  • 5
1
vote
0 answers

How to make a Kafka Kraft backup correctly?

previously, to back up kafka, it was necessary to save the state of zookeeper and the data from log.dir, now we have switched to kraft to make a backup copy of the data, now we only need to copy the contents of log.dir?
Iceforest
  • 309
  • 1
  • 11
1
vote
0 answers

Kafka KRaft controller nodes disconnect after 10 minutes

In my dev environment I have deployed a Kafka cluster in KRaft mode with three nodes that act as controller and broker. When the machines are deployed everything seems fine, but after 10 minutes a log message is displayed on node 1 that node 2 and 3…
eazy-b
  • 11
  • 2
1
vote
0 answers

kafka RaftManager Nodes disconnected frequently

im trying to deploy kafka kraft by docker-compose with bitnami image 3.3.1 and 3 nodes. but nodes disconnect frequently and i cant produce or consume messages with my python client. get docker log like this: INFO [RaftManager nodeId=2] Node 1…
jldragon
  • 11
  • 3
1
vote
0 answers

Apache Kafka Kraft QuorumControllerEventHandler processBrokerHeartbeat: unable to start processing because of NotControllerException

Kafka version 3.2.0 Running with KRaft mode i start 3 kafka brokers. The broker kafka-2 is down for long time and keep throwing error ERROR [QuorumControllerEventHandler:QuorumController@360] - [Controller 2] processBrokerHeartbeat: unable to start…
onemin
  • 11
  • 2
1
2 3