0

I try to run a Java Spring Boot application, from my host machine, which uses Kafka and that Kafka (together with zookeeper) is inside a single docker container on my host.

zookeeper.properties:

dataDir=/tmp/zookeeper
clientPort=2181
maxClientCnxns=0

server.properties:

socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=zookeeper:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=0
listeners=INTERNAL://kafka:29092,EXTERNAL://localhost:9092
advertised.listeners=INTERNAL://kafka:29092,EXTERNAL://localhost:9092
listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
inter.broker.listener.name=INTERNAL

Docker container starts simply with:

zookeeper-server-start.sh -daemon zookeeper.properties
kafka-server-start.sh server.properties

In my Spring Boot application.properties:

kafka.bootstrapAddress=localhost:9092

And configuring topic e.g.:

@Configuration
public class KafkaTopicConfig {

    @Value(value = "${kafka.bootstrapAddress}")
    private String bootstrapAddress;

    @Bean
    public KafkaAdmin kafkaAdmin() {
        Map<String, Object> configs = Map.of(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapAddress);

        return new KafkaAdmin(configs);
    }

    @Bean
    public NewTopic commandsTopic() {
        return TopicBuilder.name("testtopic").partitions(3).replicas(1).build();
    }
}

After starting application I got following error:

2022-04-02 13:04:31.430  INFO 21220 --- [           main] o.a.k.clients.admin.AdminClientConfig    : AdminClientConfig values: 
    bootstrap.servers = [localhost:9092]
    client.dns.lookup = use_all_dns_ips
    client.id = 
    connections.max.idle.ms = 300000
    default.api.timeout.ms = 60000
    metadata.max.age.ms = 300000
    metric.reporters = []
    metrics.num.samples = 2
    metrics.recording.level = INFO
    metrics.sample.window.ms = 30000
    receive.buffer.bytes = 65536
    reconnect.backoff.max.ms = 1000
    reconnect.backoff.ms = 50
    request.timeout.ms = 30000
    retries = 2147483647
    retry.backoff.ms = 100
    sasl.client.callback.handler.class = null
    sasl.jaas.config = null
    sasl.kerberos.kinit.cmd = /usr/bin/kinit
    sasl.kerberos.min.time.before.relogin = 60000
    sasl.kerberos.service.name = null
    sasl.kerberos.ticket.renew.jitter = 0.05
    sasl.kerberos.ticket.renew.window.factor = 0.8
    sasl.login.callback.handler.class = null
    sasl.login.class = null
    sasl.login.refresh.buffer.seconds = 300
    sasl.login.refresh.min.period.seconds = 60
    sasl.login.refresh.window.factor = 0.8
    sasl.login.refresh.window.jitter = 0.05
    sasl.mechanism = GSSAPI
    security.protocol = PLAINTEXT
    security.providers = null
    send.buffer.bytes = 131072
    socket.connection.setup.timeout.max.ms = 30000
    socket.connection.setup.timeout.ms = 10000
    ssl.cipher.suites = null
    ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
    ssl.endpoint.identification.algorithm = https
    ssl.engine.factory.class = null
    ssl.key.password = null
    ssl.keymanager.algorithm = SunX509
    ssl.keystore.certificate.chain = null
    ssl.keystore.key = null
    ssl.keystore.location = null
    ssl.keystore.password = null
    ssl.keystore.type = JKS
    ssl.protocol = TLSv1.3
    ssl.provider = null
    ssl.secure.random.implementation = null
    ssl.trustmanager.algorithm = PKIX
    ssl.truststore.certificates = null
    ssl.truststore.location = null
    ssl.truststore.password = null
    ssl.truststore.type = JKS

2022-04-02 13:04:31.635  INFO 21220 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka version: 3.0.0
2022-04-02 13:04:31.637  INFO 21220 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka commitId: 8cb0a5e9d3441962
2022-04-02 13:04:31.637  INFO 21220 --- [           main] o.a.kafka.common.utils.AppInfoParser     : Kafka startTimeMs: 1648897471631
2022-04-02 13:05:01.652  INFO 21220 --- [| adminclient-1] o.a.k.c.a.i.AdminMetadataManager         : [AdminClient clientId=adminclient-1] Metadata update failed

org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata

2022-04-02 13:05:01.653 ERROR 21220 --- [           main] o.springframework.kafka.core.KafkaAdmin  : Could not configure topics

Do you have any idea what possibly have gone wrong?

pizzerman
  • 23
  • 5
  • 1
    1) You've not shown your docker run command (did you forget `-p 9092:9092`?) 2) Please don't run both Zookeeper and Kafka in the same container 3) You need `EXTERNAL://0.0.0.0:9092` as a `listeners` value – OneCricketeer Apr 02 '22 at 15:51
  • 1) I was adding `-p 9092:9092`. 2) I know it is not a good thing to place Zookeeper and Kafka together but its just my small project to learn both docker and Kafka so I started with simpliest approach, at least for me. 3) Adding that value to `listeners` worked like a charm, thanks for your help. – pizzerman Apr 03 '22 at 09:06
  • The simplest approach would be to find an existing Docker Compose setup (Confluent offers many examples, including tutorials with Spring) rather than rewrite your own container – OneCricketeer Apr 03 '22 at 12:59

0 Answers0