2

I am trying to get JMX monitoring working to monitor a test kafka instance.

I have kafka (ches/kafka) running in docker via boot2docker but I can not get JMX monitoring configured properly. I have done a bunch of troubleshooting and I know the kafka instance is running properly (consumers and producers work). The issue arises when I try simple JMX tools (jconsole and jvisualvm) and both can not connect (insecure connection error, connection failed).

Configuration items of note: I connect to 192.168.59.103 (virtualbox ip found when running 'boot2docker ip') and the ches/kafka docker/kafka instance uses the port 7203 as the JMX_PORT (confirmed in the kafka startup logs). Using jconsole, I connect to 192.168.59.103:7203 and that is when the errors occur.

Any help is appreciated.

Eric Broda
  • 6,701
  • 6
  • 48
  • 72
  • 1
    Please add jvm arguments specific to JMX to the question. Did you set -Djava.rmi.server.hostname to boot2docker IP? Did you disable ssl? Did you set both `-Dcom.sun.management.jmxremote.port` and `-Dcom.sun.management.jmxremote.rmi.port` [to same value](http://stackoverflow.com/a/21552812/241986) Did you bind and expose that port in docker? Did you forward the port in boot2docker? – Boris Treukhov Jul 01 '15 at 22:38
  • As for kafka, you can use kafka protocol to send requests directly into the node inside docker to read the state of queues. Basically you send a request to read metadata, and fetch required data after the partitions are known. – Boris Treukhov Jul 01 '15 at 22:43

2 Answers2

3

For completeness, here is the solution that works:

I ran ches/kafka docker image as follows -- note that the JMX_PORT (7203) is now published appropriately:

$ docker run --hostname localhost --name kafka --publish 9092:9092 --publish 7203:7203 --env EXPOSED_HOST=192.168.59.103 --env ZOOKEEPER_IP=192.168.59.103 ches/kafka

Also, the following environment is set in the kafka-run-class.sh (.bat for windows)

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

But I needed to add one additional item (thanks to one of the commenters for pointing this out):

-Dcom.sun.management.jmxremote.rmi.port=7203

Now, to run the ches/docker image in boot2docker you just need to set one of the recognized environment variables (KAFKA_JMX_OPTS or KAKFA_OPTS) to add the additional item and it now works.

Thanks for the help!

Eric Broda
  • 6,701
  • 6
  • 48
  • 72
  • Owner of the image in question here—sorry that you had the trouble, forcing the JVM not to use random high ports for RMI with that additional property setting was indeed the key missing piece for making JMX more usable. [This should be fixed](https://github.com/ches/docker-kafka/commit/d4d974e8dcf9a2ba4b4dc2cd46389e63eaf95e08) and it's in the latest tag `ches/kafka:0.8.2.1`, let me know if that's smoother to work with! – ches Aug 24 '15 at 17:59
  • No problem -- the work you did to get the image setup was great and saved me a ton of time. Your solution seems to works for me. Thanks! – Eric Broda Aug 24 '15 at 21:53
0

There's no reason the kafka docker port would bind to the same port in the boot2docker VM except if you specify it.

Try running it with -p 7203:7203 to force the 1:1 forwarding of the port.

C4stor
  • 8,355
  • 6
  • 29
  • 47