4

I am trying to start the Kafka with KRaft by following doc here - https://kafka.apache.org/quickstart on windows machine.

Note - The same version of kafka works perfectly fine with zookeeper

Step 1 - generate uuid (.\bin\windows\kafka-storage.bat random-uuid)

Step 2 - format log dirs (.\bin\windows\kafka-storage.bat format -t d1hZRwFeRuqyNTTwR9grGw -c .\config\kraft\server.properties)

Step 3 - Start the server (.\bin\windows\kafka-server-start.bat .\config\kraft\server.properties)

But it's not starting and giving the error -

    [2023-03-26 23:34:36,921] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2023-03-26 23:34:37,252] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2023-03-26 23:34:37,466] INFO Starting controller (kafka.server.ControllerServer)
[2023-03-26 23:34:37,866] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2023-03-26 23:34:37,926] INFO Awaiting socket connections on 0.0.0.0:9093. (kafka.network.DataPlaneAcceptor)
[2023-03-26 23:34:37,983] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Created data-plane acceptor and processors for endpoint : ListenerName(CONTROLLER) (kafka.network.SocketServer)
[2023-03-26 23:34:37,994] INFO [SharedServer id=1] Starting SharedServer (kafka.server.SharedServer)
[2023-03-26 23:34:38,111] INFO [LogLoader partition=__cluster_metadata-0, dir=E:\tmp\kraft-combined-logs] Loading producer state till offset 0 with message format version 2 (kafka.log.UnifiedLog$)
[2023-03-26 23:34:38,113] INFO [LogLoader partition=__cluster_metadata-0, dir=E:\tmp\kraft-combined-logs] Reloading from producer snapshot and rebuilding producer state from offset 0 (kafka.log.UnifiedLog$)
[2023-03-26 23:34:38,116] INFO [LogLoader partition=__cluster_metadata-0, dir=E:\tmp\kraft-combined-logs] Producer state recovery took 2ms for snapshot load and 0ms for segment recovery from offset 0 (kafka.log.UnifiedLog$)
[2023-03-26 23:34:38,175] INFO Initialized snapshots with IDs SortedSet() from E:\tmp\kraft-combined-logs\__cluster_metadata-0 (kafka.raft.KafkaMetadataLog$)
[2023-03-26 23:34:38,204] INFO [raft-expiration-reaper]: Starting (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper)
[2023-03-26 23:34:38,423] ERROR [SharedServer id=1] Got exception while starting SharedServer (kafka.server.SharedServer)
java.io.UncheckedIOException: Error while writing the Quorum status from the file E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state
        at org.apache.kafka.raft.FileBasedStateStore.writeElectionStateToFile(FileBasedStateStore.java:155)
        at org.apache.kafka.raft.FileBasedStateStore.writeElectionState(FileBasedStateStore.java:128)
        at org.apache.kafka.raft.QuorumState.transitionTo(QuorumState.java:477)
        at org.apache.kafka.raft.QuorumState.initialize(QuorumState.java:212)
        at org.apache.kafka.raft.KafkaRaftClient.initialize(KafkaRaftClient.java:369)
        at kafka.raft.KafkaRaftManager.buildRaftClient(RaftManager.scala:240)
        at kafka.raft.KafkaRaftManager.<init>(RaftManager.scala:166)
        at kafka.server.SharedServer.start(SharedServer.scala:228)
        at kafka.server.SharedServer.startForController(SharedServer.scala:128)
        at kafka.server.ControllerServer.startup(ControllerServer.scala:188)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:98)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:98)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:98)
        at kafka.Kafka$.main(Kafka.scala:115)
        at kafka.Kafka.main(Kafka.scala)
Caused by: java.nio.file.FileSystemException: E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state.tmp -> E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state: The process cannot access the file because it is being used by another process
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:403)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:293)
        at java.base/java.nio.file.Files.move(Files.java:1430)
        at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:949)
        at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:932)
        at org.apache.kafka.raft.FileBasedStateStore.writeElectionStateToFile(FileBasedStateStore.java:152)
        ... 15 more
        Suppressed: java.nio.file.FileSystemException: E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state.tmp -> E:\tmp\kraft-combined-logs\__cluster_metadata-0\quorum-state: The process cannot access the file because it is being used by another process
                at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
                at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
                at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:317)
                at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:293)
                at java.base/java.nio.file.Files.move(Files.java:1430)
                at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:946)
                ... 17 more
[2023-03-26 23:34:38,430] ERROR [ControllerServer id=1] Fatal error during controller startup. Prepare to shutdown (kafka.server.ControllerServer)
java.lang.NullPointerException: Cannot invoke "kafka.raft.KafkaRaftManager.apiVersions()" because the return value of "kafka.server.SharedServer.raftManager()" is null
        at kafka.server.ControllerServer.startup(ControllerServer.scala:197)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:98)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:98)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:98)
        at kafka.Kafka$.main(Kafka.scala:115)
        at kafka.Kafka.main(Kafka.scala)
[2023-03-26 23:34:38,434] INFO [ControllerServer id=1] shutting down (kafka.server.ControllerServer)
[2023-03-26 23:34:38,435] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Stopping socket server request processors (kafka.network.SocketServer)
[2023-03-26 23:34:38,442] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Stopped socket server request processors (kafka.network.SocketServer)
[2023-03-26 23:34:38,444] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Shutting down socket server (kafka.network.SocketServer)
[2023-03-26 23:34:38,478] INFO [SocketServer listenerType=CONTROLLER, nodeId=1] Shutdown completed (kafka.network.SocketServer)
[2023-03-26 23:34:38,480] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
java.lang.NullPointerException: Cannot invoke "kafka.raft.KafkaRaftManager.apiVersions()" because the return value of "kafka.server.SharedServer.raftManager()" is null
        at kafka.server.ControllerServer.startup(ControllerServer.scala:197)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:98)
        at kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:98)
        at scala.Option.foreach(Option.scala:437)
        at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:98)
        at kafka.Kafka$.main(Kafka.scala:115)
        at kafka.Kafka.main(Kafka.scala)
[2023-03-26 23:34:38,488] INFO App info kafka.server for 1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Arvind Kumar
  • 459
  • 5
  • 21

1 Answers1

0

Run the commands with administrative privileges, and you'll be fine. Note: In case of WSL2, run the commands with sudo.

ozeray
  • 2,134
  • 2
  • 18
  • 13