0

I wanted to create a docker-compose for zookeeper/kafka for integration test purposes.

  zookeeper:
    image: repository/zookeeper:3.8
    ports:
      - 2181:2181
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
      - ZOOKEEPER_SASL_ENABLED=false

  kafka:
    image: repository/kafka:3.0.1
    ports:
      - 9092:9092
    environment:
      - ALLOW_PLAINTEXT_LISTENER=yes
      - ZOOKEEPER_SASL_ENABLED="false"
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_ADVERTISED_LISTENERS=INTERNAL://kafka:9093,CLIENT://:9092
      - KAFKA_CFG_LISTENERS=INTERNAL://:9093,CLIENT://:9092
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,CLIENT:SASL_SSL
      - KAFKA_CFG_LISTENER_NAME_INTERNAL_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=INTERNAL
      - KAFKA_CFG_SECURITY_PROTOCOL=SASL_SSL
      - KAFKA_CFG_TLS_TYPE=JKS
      - KAFKA_CFG_SASL_ENABLED_MECHANISMS=PLAIN,SCRAM-SHA-512
      - KAFKA_CFG_SASL_MECHANISM=SCRAM-SHA-512
      - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=SCRAM-SHA-512
      - SECURITY_INTER_BROKER_PROTOCOL=SASL_SSL
      - KAFKA_CFG_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=
      - KAFKA_CFG_SSL_KEYSTORE_PASSWORD=password.1
      - KAFKA_CFG_SSL_KEY_PASSWORD=password.1
      - KAFKA_CFG_SSL_TRUSTSTORE_PASSWORD=password.1
      - KAFKA_CERTIFICATE_PASSWORD=password.1
      - KAFKA_SSL_CLIENT_AUTH=required
      - KAFKA_OPTS=-Djava.security.auth.login.config=/etc/kafka/jaas/kafka_jaas.conf
    volumes:
      - './src/test-integration/resources/certs/tst/kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks:ro'
      - './src/test-integration/resources/certs/tst/kafka.truststore.jks:/opt/bitnami/kafka/config/certs/kafka.truststore.jks:ro'
      - './src/test-integration/resources/jaas/kafka_jaas.conf:/etc/kafka/jaas/kafka_jaas.conf:ro'
    depends_on:
      - zookeeper

kafka_jaas.conf:

KafkaClient {
  org.apache.kafka.common.security.scram.ScramLoginModule required
  username="user"
  password="password"
};
Client{};

With this server configuration I'm trying to connect from my Spring application with properties:

spring:
  kafka:
    properties:
      sasl:
        mechanism: SCRAM-SHA-512
        jaas:
          config: org.apache.kafka.common.security.scram.ScramLoginModule
            required username='user' password='password';

I only want to use SCRAM for my client app, not inter-broker communication. What here can be wrong if I receive the exception with the message?

Connection to node -1 (localhost/127.0.0.1:9092) failed authentication due to: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-512
OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
yaw
  • 303
  • 2
  • 11
  • Would like to point out that `KAFKA_CFG_ADVERTISED_LISTENERS` is missing `localhost` on port 9092. You should also remove `KAFKA_CFG_SECURITY_PROTOCOL` if you only want it in one listener. And how are you providing the key/trust store to the Spring app for TLS? – OneCricketeer Aug 10 '22 at 14:37
  • Also by Spring properties: `ssl: trust-store-location: classpath:certs/tst/kafka.truststore.jks trust-store-password: server.1 trust-store-type: JKS` – yaw Aug 10 '22 at 16:31
  • When the app starts though, do you see those properties given to the Kafka config? Can you show the logs printed from the kafka configs? – OneCricketeer Aug 11 '22 at 13:35

0 Answers0