Stack Overflow
Stack Overflow

Questions tagged [apache-kafka-security]

The following security measures are currently supported:

  1. Authentication of connections to brokers from clients (producers and consumers), other brokers and tools, using either SSL or SASL. Kafka supports the following SASL mechanisms:
    • SASL/GSSAPI (Kerberos) - starting at version 0.9.0.0
    • SASL/PLAIN - starting at version 0.10.0.0
    • SASL/SCRAM-SHA-256 and SASL/SCRAM-SHA-512 - starting at version 0.10.2.0
    • SASL/OAUTHBEARER - starting at version 2.0
  2. Authentication of connections from brokers to ZooKeeper
  3. Encryption of data transferred between brokers and clients, between brokers, or between brokers and tools using SSL (Note that there is a performance degradation when SSL is enabled, the magnitude of which depends on the CPU type and the JVM implementation.)
  4. Authorization of read / write operations by clients
  5. Authorization is pluggable and integration with external authorization services is supported

Links

Related Tags

55 questions
12
votes
2 answers

No serviceName defined in either JAAS or Kafka config (not Kerberos)

I'm trying to configure a kafka client to authenticate against a secure kafkaserver. I've set up the jaas and ssl configs, but it's complaining about serviceNames. I am not using…
James Liu
  • 196
  • 1
  • 2
  • 10
11
votes
4 answers

KAFKA and SSL : java.lang.OutOfMemoryError: Java heap space when using kafka-topics command on KAFKA SSL cluster

this is my first post on Stackoverflow, i hope i didnt choose the wrong section. Context : Kafka HEAP size is configured on following file : /etc/systemd/system/kafka.service With following parameter : Environment="KAFKA_HEAP_OPTS=-Xms6g…
Blyyyn
  • 191
  • 1
  • 2
  • 9
7
votes
1 answer

No JAAS configuration section named 'Server' was foundin '/kafka/kafka_2.12-2.3.0/config/zookeeper_jaas.conf'

when i run the zookeeper from the package in the kakfa_2.12-2.3.0 i am getting the following error $ export KAFKA_OPTS="-Djava.security.auth.login.config=/kafka/kafka_2.12-2.3.0/config/zookeeper_jaas.conf" $ ./bin/zookeeper-server-start.sh …
RamanaMuttana
  • 481
  • 6
  • 22
6
votes
2 answers

Apache kafka consumer java.security.cert.CertificateException: No subject alternative names present

I'm trying to connect to remove kafka broker as consumer using 0.11.0.3 kafka version via SSL using commandline tool, connection string is the following kafka-console-consumer.bat \ --bootstrap-server host:port \ --topic topicName \ …
Ilya Melnik
  • 63
  • 1
  • 3
6
votes
1 answer

librdkafka consumer and ssl configuration

I'm using librdkafka as a client consumer, and I have configured the broker and the client to support SSL, for the broker: listeners =…
Ismail
  • 2,322
  • 1
  • 12
  • 26
5
votes
1 answer

Java Producer/Consumer kafka client properties required when accessing a SSL-Auth secured Kafka brokers/cluster?

When we connect to Kafka cluster/kafka, In the java clients we define certain properties - Example Producer Properties Properties props = new Properties(); props.put("bootstrap.servers", "localhost:9092"); props.put("acks", "all"); …
samshers
  • 1
  • 6
  • 37
  • 84
4
votes
1 answer

Strimzi operator Kafka cluster ACL not enabling with type: simple

We know to enable Kafka ACL property authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer to be added in server.properties but how to enable that if Kafka cluster is running by Strimzi operator? From Strimzi documents I have come to know in…
sudhir tatraju
  • 93
  • 1
  • 7
4
votes
1 answer

Kafka java consumer SSL handshake Error : java.security.cert.CertificateException: No subject alternative names present

I'm running kafka 2.13-2.4.1 and configuring an SSL connection between kafka client (consumer) written in java and a kafka cluster (3 nodes with each node having one broker). I used the official documentation through Confluent's Documentation which…
Sami Issam
  • 43
  • 1
  • 1
  • 3
3
votes
0 answers

Kafka Schema Registry failed to initialize after ZK and Brokers are configured with SASL_PLAINTEXT Security

We are using Confluent community edition setup for Kafka, currently we have a requirement to configure ACLs around the cluster, accordingly we have configured the zk and broker nodes so clients requires authentication(username/password)…
Amol Fasale
  • 942
  • 1
  • 10
  • 32
3
votes
1 answer

How to setup proxy layer on kafka broker?

I am trying to setup a proxy server (preferred in java) that can pass my connection stream to kafka broker. Client (Consumer/Producer) <--> [PROXY SERVER] <--> Kafka Broker My use case is consume the data from kafka, but the configuration params…
mr-woot
  • 88
  • 1
  • 10
3
votes
0 answers

ERROR:kafka.client:Unable to bootstrap from kafka

Here is my consumer config. I am using SSL for authentication. consumer = KafkaConsumer(config['kafka_topic'], bootstrap_servers=config['kafka_brokers'], auto_offset_reset='earliest', …
Sunny
  • 14,522
  • 15
  • 84
  • 129
3
votes
1 answer

Kafka Avro Console Consumer not working after enabling SSL encryption and authentication

I have recently enabled 2-way authentication on my Kafka Cluster. I am trying to consume messages from a topic in Avro format using kafka-avro-console-consumer --bootstrap-server kafka-host:9092 --topic my_topic --consumer.config…
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156
2
votes
1 answer

Zookeeper: cnxn.saslServer is null and Kafka:the quorum member's saslToken is null

To provide access to only kafka for creating, deleting topics, I am creating a plaintext SASL security between kafka and zookeeper. I get the following error and can not figure out why. zookeeper_1 | 2020-07-20 10:19:06,907 [myid:] - ERROR…
Imam Bux
  • 1,006
  • 11
  • 27
2
votes
1 answer

How to setup Kafka Security and listeners in dynamic environment?

These resources provide a great view into how to setup kafka security (encryption + ssl) and good idea of kafka listeners. setting up ssl auth listeners and advertised-listeners: a, b, c, d(from confluent guy - Robin M) But this seems to work…
samshers
  • 1
  • 6
  • 37
  • 84
2
votes
1 answer

Only allowing consumers from certain hosts/IPs access to kafka topics

We have a Kafka-based system, and a large team of devs who are working on this system. We are still in development and testing phase, and have not gone to production yet. For majority of local dev testing, we do not need to consume Kafka messages,…
theckler
  • 21
  • 3
1
2 3 4