1

I am working on an application which have following flow :

KAFKA PRODUCER --> SPARK STREAMING APPLICATION (KAFKA CONSUMER) --> HBASE --> PYTHON VIZ.

I have Spark, Kafka 2.3.0, HBase 1.3.5 and Zookeeper 3.4.13 installed. Spark, Kafka and Zookeeper are working successfully.

I'm having difficulty connecting HBase to Zookeeper.

  1. I am using HBase in pseudo distributed mode. Below is my hbase- site.xml :
<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://localhost:9000/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/usr/local/var/zookeeper</value>
  </property>
  <property>
     <name>hbase.cluster.distributed</name>
     <value>true</value>
  </property>
</configuration>

  1. I have set HBASE_MANAGES_ZK=false in hbase-env.sh as I am running Zookeeper server separately from kafka/bin/zookeeper-server- start .

  2. jps command output:

4656 QuorumPeerMain
2977 HRegionServer
6083 NameNode
6694 Jps
6330 SecondaryNameNode
6524 ResourceManager
6191 DataNode

I am following below steps :

  • Starting hadoop
  • Starting Zookeeper server
    kafka/bin/zookeeper-server-start /usr/local/Cellar/apache-zookeeper-3.5.5-bin/conf/zoo.cfg
  • Running KAFKA Producer and consumer application.

In HBase shell the list command returns the following error :

Error - Can't get master address from ZooKeeper; znode data == null

I am stuck at this point. Please help me to understand that what I am missing.

Christian
  • 4,902
  • 4
  • 24
  • 42
blueWings
  • 71
  • 1
  • 9
  • Potential duplicate: https://stackoverflow.com/questions/22663484/get-error-cant-get-master-address-from-zookeeper-znode-data-null-when-us – Christian Sep 03 '19 at 22:51
  • @Christian Hey , Unfortunately the posted link didn't answer my problem :) – blueWings Sep 03 '19 at 22:56
  • You might want to read some code, seems that there is a zookeeper.znode.master which registers an address which seems pretty important. https://github.com/apache/hbase/blob/025ddce868eb06b4072b5152c5ffae5a01e7ae30/hbase-zookeeper/src/main/java/org/apache/hadoop/hbase/zookeeper/MasterAddressTracker.java#L156 – Christian Sep 03 '19 at 23:50
  • I assume a restart does not help. Have you got more logs or is the error the only message. Looking in some forums I have seen that many many people have or had the same issue / error message. I hope there is a solution for you. – Christian Sep 06 '19 at 22:52
  • @Christian for the time being, I am storing data in MongoDB. I will work on this problem after completing this project and will update the solution. Thank you for your help. – blueWings Sep 07 '19 at 21:42
  • There are many many answers in the link posted on top of the comments. Maybe you should start here: Look into the hbase log in /var/log/hbase to find out what is causing the issue. – Christian Sep 11 '19 at 09:06

0 Answers0