4
{
   "mdc":{
      
   },
   "timestamp":"2021-05-11 11:48:04.055",
   "level":"ERROR",
   "logger":"org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner",
   "message":"Failed to create topics",
   "exception":"\"\norg.apache.kafka.common.errors.UnsupportedVersionException: Creating topics with default partitions/replication factor are only supported in CreateTopicRequest version 4+. The following topics need values for partitions and replicas:"

Please suggest what changes are required as i am getting this error.

James Z
  • 12,209
  • 10
  • 24
  • 44
shagun
  • 63
  • 1
  • 5

4 Answers4

4

I see you are new here. You should always include version information and full stack trace for questions like this.

Upgrade your broker to >= 2.4 or set the binder replication factor property.

See https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/commit/4161f875ede0446ab1d485730c51e6a2c5baa37a

  • Change default replication factor to -1

Binder now uses a default value of -1 for replication factor signaling the broker to use defaults. Users who are on Kafka brokers older than 2.4, need to set this to the previous default value of 1 used in the binder.

In either case, if there is an admin policy that requires replication factor > 1, then that value must be used instead.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Using Kafka Version 2.6.0 but still getting this issue – shagun May 14 '21 at 07:31
  • That's the client version; the broker needs to be 2.4 or later; set the binder property to the replication factor you want. – Gary Russell May 14 '21 at 13:23
  • Thanks Gary, added binder property to the replication factor "1" it works... But previously we are using spring-cloud-stream-binder-kafka.version "3.0.7" at that time issue didn't occurs , currently we are using "3.1.2" and issue occurs. Is it because of replication factor changes from "1" to "-1” ? So, we need to explicitly add the replication factor property. – shagun May 17 '21 at 03:47
  • Yes, it was a new feature added to Kafka Brokers at 2.4; users complained that SCSt overrode that behavior and forced them to configure it in both places. In order to satisfy those users, SCSt 3.1 changed its default to -1 so the broker configuration would win. As the documentation stated, this meant that users with old brokers would have to change it to the old default. Price of progress. – Gary Russell May 17 '21 at 13:11
3

Overriding the default replication factor(-1) with a non negative value fixed the issue for me.

spring.cloud.stream.kafka.binder.replication-factor=1

For application.yaml file use:

spring.cloud.stream.kafka.binder.replicationFactor: 1
Roee Gavirel
  • 18,955
  • 12
  • 67
  • 94
javachipsy
  • 396
  • 3
  • 6
0

Change the code to following . The fluent api allows to give partitions and replicas.

@Bean
    NewTopic topicBytes() {
        return TopicBuilder.name("reflectoring-bytes").partitions(1).replicas(1).build();
    }
Tanmay Patil
  • 659
  • 1
  • 4
  • 21
0

I had the same issue. And the following link helped me to solve it.

https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1134

The property path for cloud kafka stream is different. Instead of spring.cloud.stream.kafka.binder we have to use spring.cloud.stream.kafka.streams.binder

rajkumar21
  • 183
  • 3
  • 9