0

I am trying to run a second instance for Kafka Rest. Here's the properties file for the first one:

id=kafka-rest-test-server-1
schema.registry.url=http://localhost:8081,http://localhost:9081
zookeeper.connect=localhost:2181,localhost:2182,localhost:2183
bootstrap.servers=PLAINTEXT://localhost:9092,PLAINTEXT://localhost:9093,PLAINTEXT://localhost:9094

According to Confluent's docs:

The only requirement for multiple instances is that you set a unique id for each instance.

However, when I run the second instance with other id, an exception is thrown mentioning that the address is already in use: 

WARN FAILED io.confluent.rest.Application$1@41294f8: java.net.BindException: Address already in use (org.eclipse.jetty.util.component.AbstractLifeCycle:212)
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)

...
Giorgos Myrianthous
  • 36,235
  • 20
  • 134
  • 156

1 Answers1

1

The ID doesn't control the port. "Multiple instances" in the documentation means independent machines

Just like the Schema Registry, configure the listeners

listeners
Comma-separated list of listeners that listen for API requests over either HTTP or HTTPS

Default: “http://0.0.0.0:8082“

https://docs.confluent.io/current/kafka-rest/docs/config.html#kafkarest-config

port property is deprecated, but if you're running an old version, wouldn't hurt to set it as well

For example, first instance, leave the defaults

Second instance, use any available port 

listeners=http://0.0.0.0:18082

It is generally recommended that you use more than one machine for high availability and distributed processing. Running 3 Zookeepers, 3 Brokers, 2 Registries, and 2 Proxies, you're basically begging to run out of RAM

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Just to make it clear, say that I want to run two kafka connect instances on http://0.0.0.0:8082 and http://1.1.1.1:8082, should I place both listenes on both files e.g. `listeners=http://0.0.0.0:8082,http://1.1.1.1:8082` or should I place only one listener per instance? – Giorgos Myrianthous Jun 28 '18 at 08:36
  • 1
    Assuming you mean REST Proxies. It's a bind address. You can list as many as you want, but anything other than 0.0.0.0 will take precedence and limit connectivity. See general description at https://stackoverflow.com/questions/3552680/bind-address-and-mysql-server My point was to change the port number – OneCricketeer Jun 28 '18 at 08:41