0

I am getting the exception below while trying to connect HBase from spark2 and insert some data using the HBaseConfiguration.

18/03/05 15:18:47 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration

I read on Cloudera that Spark2 On HBase is not Supported as Spark On HBase is a CDH component that has a dependency on Spark 1.6. Is there any workaround to insert data into HBase from Spark2?

yAsH
  • 3,367
  • 8
  • 36
  • 67
  • You're just missing a dependency... Not exactly a problem with Spark 2. How are you running the code? Are you adding any external packages? – OneCricketeer Mar 09 '18 at 01:15
  • I am running the code on the Yarn cluster. I added this configuration (--conf spark.driver.extraClassPath="/etc/hbase/conf" --conf spark.yarn.security.tokens.hbase.enabled=false) in the spark properties. Also, writing to Hbase was working fine on Spark 1.6. – yAsH Mar 09 '18 at 15:06
  • Okay, well according to here, there is no such available connector for Spark2 yet. https://stackoverflow.com/questions/40908891/which-hbase-connector-for-spark-2-0-should-i-use – OneCricketeer Mar 09 '18 at 17:02

0 Answers0