3

I will start telling by what i am trying to achieve. So My setup is:

  1. 6 VM's running Ubuntu 14.04 Version. - In 3 of them i have setup Kafka and in 3 i have created the zookeeper instance.

  2. I start the produce and consume and all seems to work fine no issue.

Now i want to secure the setup using the SSL which has come up with kafka 0.9 version. I only want to setup SSL between client and kafka broker so that they can communicate securely. I followed the folowing link.

The only change i have done is: i have replaced the localhost with the IP of the kafka broker as i have 6 VM's running on same network.

Where i am getting stuck is after generating the the certificates when i run the following command:

openssl s_client -debug -connect localhost:9093 -tls1

I get the message connect:

Connection refused connect:errno=111

I dont know how to proceed with this. I tried googling going through the documents but couldn't fine anything.

Also some suggestion what will be the ideal setup for security, my req is i only need secure communication between kafka client and brokers, i dont need to worry about kafka-to-kafka and kafka-to-zookeeper communication.

Please help, Vishesh.

Murmel
  • 5,402
  • 47
  • 53
Vishesh
  • 3,599
  • 4
  • 22
  • 36

2 Answers2

2

After few tries and help from some colleagues i was able to get the entire setup working correctly over the 9093 port.

What went wrong was with the certificate signing. As i had 3 kafka brokers so i had created 3 different CA to sign the certificate which was wrong.

What needs to be done is you need to have the certificate signed by the same single CA in all the brokers and clients, Or in case you have different certificate or you plan to use different certificates for the brokers the you need to have a root CA which signs all the other CA.

You can read more about it in Hierarchy of Trust over here :- https://msdn.microsoft.com/en-us/library/windows/desktop/aa382479(v=vs.85).aspx

So if you have done the setup in one broker user the same ca-cert file in all other brokers and client to sign the certificate and it will work .!!

Vishesh
  • 3,599
  • 4
  • 22
  • 36
  • makes sense. i am also trying to do something similar. can you please take a look at my question http://stackoverflow.com/questions/36751542/kafka-cannot-set-up-access-control-list-with-two-brokers-on-same-machine and see if it makes sense. i used the same root ca to sign all the certificates but i still get the error – AbtPst Apr 20 '16 at 18:14
0

Check if the Corporate ports which are used for SSL are open. When we did set up this after long research it was this simple thing.

Paresh
  • 564
  • 6
  • 23
  • Paresh thanks for replying ... Sorry but i am new to linux env. I tried using netstat to see it. I see tcp port over there. If i am correct Kafka uses 9093 for SSl setup. Can you help me how to do that? – Vishesh Feb 27 '16 at 17:24