1

We have a java application running on the Liberty IBM WebSphere server and trying to connect to the HBase on the HDP cluster to persist some data.

Now we are facing issues to connect to HBase(kerberized) on HDP cluster.

We have been able to connect to HBase via Spark, Storm or application running with in the cluster but facing issue as we are sitting outside the cluster.

We have tried multiple approaches and followed these links https://community.hortonworks.com/articles/120858/connecting-to-kerberos-secured-hbase-cluster-from.html

https://community.hortonworks.com/articles/48831/connecting-to-hbase-in-a-kerberos-enabled-cluster.html

So basically, we have copied the conf from the Hbase (hbase-site.xml, hdfs-site.xml and core-site.xml) to our application classpath and plus copied the Keytab for our service account user. We tried 4 different approachs

1)

Used hbase.zookeeper.quorum and hbase.zookeeper.property.clientPort

and

our service account user principle name and Keytab forUserGroupInformation.loginUserFromKeytab(principal, keytabLocation);

2)

Used hbase.zookeeper.quorum and hbase.zookeeper.property.clientPort

and

HBASE master principle name and Keytab forUserGroupInformation.loginUserFromKeytab(principal, keytabLocation);

3)

Used hbase-site.xml, hdfs-site.xml and core-site.xml

and our

service account user principle name and Keytab forUserGroupInformation.loginUserFromKeytab(principal, keytabLocation)

;

4)

Used hbase-site.xml, hdfs-site.xml and core-site.xml clientPort

and

HBASE master service account user principle name and Keytab forUserGroupInformation.loginUserFromKeytab(principal, keytabLocation);

Attached the below code snippet

    public Connection getHBaseConnection() throws IOException, InterruptedException {
    final Configuration configuration = HBaseConfiguration.create();
//configuration.set(HBASE_ZOOKEEPER_PROPERTY_CLIENT_PORT, environment.getProperty(HBASE_ZOOKEEPER_PROPERTY_CLIENT_PORT));
    //configuration.set(HBASE_ZOOKEEPER_QUORUM, environment.getProperty(HBASE_ZOOKEEPER_QUORUM));
    //configuration.set(ZOOKEEPER_ZNODE_PARENT, environment.getProperty(ZOOKEEPER_ZNODE_PARENT)); */

    configuration.addResource(getClass().getResourceAsStream(CORE_SITE_PATH));
    configuration.addResource(getClass().getResourceAsStream(HBASE_SITE_PATH));
    configuration.addResource(getClass().getResourceAsStream(HDFS_SITE_PATH));

    configuration.set("hadoop.security.authentication", "kerberos");
    configuration.set("hbase.security.authentication", "kerberos");
    configuration.set("hbase.cluster.distributed", "true");
    configuration.set("hbase.rpc.protection", "authentication");

//System.setProperty("java.security.auth.login.config", "src/main/resources/sbx/hbase_client_jaas.conf");
//System.setProperty("java.security.krb5.conf","src/main/resources/sbx/krb5.conf");
//System.setProperty("sun.security.krb5.debug", "false");
//System.setProperty("java.security.krb5.realm", "HDP.SANDBOX.LOCAL");
//System.setProperty("java.security.krb5.kdc", "shared-serverbox-01.sandbox.local");


    configuration.set("hbase.master.kerberos.principal", "hbase/_HOST@HDP.SANDBOX.LOCAL");
    configuration.set("hbase.master.keytab.file", "src/main/resources/sbx/hbase.service.keytab");
    configuration.set("hbase.regionserver.kerberos.principal", "hbase/_HOST@HDP.SANDBOX.LOCAL");
    configuration.set("hbase.regionserver.keytab.file", "src/main/resources/sbx/hbase.service.keytab");


    String keyTab = "src/main/resources/pasusr.keytab";
    String principle = environment.getProperty(PRINCIPAL);

    String keyTabHbase = "src/main/resources/sbx/hbase.service.keytab" ;
    String principleHbase = "hbase/shared-serverbox-01.sandbox.localT@HDP.SANDBOX.LOCAL";

    UserGroupInformation.setConfiguration(configuration);
    UserGroupInformation ugi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principleHbase, keyTabHbase);
    UserGroupInformation.setLoginUser(ugi);
    return ugi.doAs(new PrivilegedExceptionAction<Connection>() {
        @Override
        public Connection run() throws IOException {
            Connection connection = ConnectionFactory.createConnection(configuration);
            System.out.println("Connected " + connection);
            return connection;
        }
    });

}
Puneet Babbar
  • 105
  • 1
  • 1
  • 7
  • I would definitively recommend using KNOX to connect to HBase from outside your HDP cluster (see https://knox.apache.org/books/knox-0-9-0/user-guide.html) – Harold Jun 18 '18 at 09:02
  • Did you check https://stackoverflow.com/questions/44265562/spark-on-yarn-secured-hbase ? – Samson Scharfrichter Jun 18 '18 at 18:07
  • @Harold : KNOX is not available. – Puneet Babbar Jun 18 '18 at 20:41
  • @SamsonScharfrichter Thanks the link has a good explanation, but we are not using Spark here. It is a java based web application, and trying to connect to HBase. – Puneet Babbar Jun 18 '18 at 20:43
  • In the post, the `KRB_DEBUG_OPTS` env var contained `-Dsun.security.krb5.debug=true -Djava.security.debug=gssloginconfig,configfile,configparser,logincontext` which is the first step for Kerberos debugging. Also `export HADOOP_JAAS_DEBUG=true` which is Hadoop-specific. Now **get to work**. And you if you really "have issues", then **show the error messages** - we are not magicians. – Samson Scharfrichter Jun 19 '18 at 06:53
  • @SamsonScharfrichter: I agree that Kerberos is not so complicated. My comment was rather about solution architecture and security. Basically an “external” application may not be able to reach all cluster data nodes, especially if you need to setup special network rules or permissions. And you may face some issues if you add new data anodes to the clusters and forget to update those rules. If the application is just co-hosted (eg running on a edge node), there’s indeed no point of using KNOX. – Harold Jun 19 '18 at 08:29
  • @PuneetBabbar It looks like the problem is related to “outside the cluster” so the points to check are: ability to connect to Kerberos server and get the ticket, connectivity with all data nodes – Harold Jun 19 '18 at 08:30

0 Answers0