0

I am trying to configure an hbase client to connect to an HBase that is running in a docker container. Given it is for integration tests purposes, we are fine to put the master and regionserver hardcoded (all of them to the same address).

We use a cloudera distribution, but I guess this should be the same. I have defined the "hbase.master" in the org.apache.hadoop.conf.Configuration

configuration.set("hbase.master", config.getMaster());

and it seems to be accepted (looked other questions here in SO that were mentioning this).

But trying to do similar for the regionserver as explained in http://archive.cloudera.com/cdh5/cdh/5/hbase-0.98.6-cdh5.3.4/book/config.files.html

configuration.set("hbase.regionserver.port", config.getRegionserver());

and I can see that still this is retrieved from Zookeeper as the error shows failure to connect with a socket to 60002 which is the regionserver portnumber (the one I put in the config is a different one which is declared in Docker.

user73540
  • 1
  • 2
  • Not sure if it helps, but to give a bit of context, we are using testcontainers and we are also spinning a SocatContainer instance as they do here https://github.com/testcontainers/testcontainers-java/blob/1.11.2/modules/kafka/src/main/java/org/testcontainers/containers/KafkaContainer.java Doing that kind of redirection is working but Hbase client is stubborn and only takes what ZK tells him to take – user73540 Jul 14 '20 at 09:28
  • Not sure I understand the problem correctly, but why does your client need to connect to Master and region server? You need to be connecting to the zooekeper quorum. Wouldn't be surprised if in your test environment these happen to be the same machine, but technically they are distinct processes listening on distinct ports. – VS_FF Jul 15 '20 at 15:23
  • the hbase client needs to be configured only with the ZK quorum, indeed. But internally, the client will make connections to the master and regionserver according to the hbase properties that Zookeeper will have. This is only a situation that matters in e2e tests or integration tests – user73540 Jul 16 '20 at 14:27

1 Answers1

0

A first and simpler to put the entry in /etc/hosts so the hbase server name points to the loopback address.

As we did not want to amend config files in our CI environment, A workaround we have found is to setup our own hostname resolver, based on this answer https://stackoverflow.com/a/43870031/3774176

Only works in JDK8, but beyond that, there is a similar solution.

user73540
  • 1
  • 2