0

I am reading a HBase table via Phoenix in spark-scala, I am able to read a column without a peroid/dot(.)

But when I try to read a column with dot ex: column name "first.name", it throws below error. but I can read data from phoenix shell without issues.

How can I read HBase table in spark via phoenix with column name having "dot" i.e. first.name, last.name etc

Error: Exception in thread "main" org.apache.phoenix.schema.ColumnFamilyNotFoundException: ERROR 1001 (42I01): Undefined column family. familyName=xyz

Any help is greatly appreciated.

Thanks!

AnumNuma
  • 31
  • 3
  • Does this answer your question? [Column name with dot spark](https://stackoverflow.com/questions/44367019/column-name-with-dot-spark) – mck Feb 20 '21 at 08:03
  • Still it error out as Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot resolve column name "`"first.name"`" among (ROWKEY, first.name. I tried multiple different options, but no luck. – AnumNuma Feb 21 '21 at 03:14
  • @AnumNuma Could you add the piece of code that generates that exception in your question? – blackbishop Feb 21 '21 at 11:47
  • mapped a view in phoenix on top of the table "personal_data", working fine in phoenix. CREATE view "personal_data" ( rowkey VARCHAR PRIMARY KEY , "personal"."first.name" VARCHAR, "personal"."last.name" VARCHAR ); val df = spark.sqlContext.read.format("org.apache.phoenix.spark").option("table", "\"personal_data\"").option("zkUrl", zkurl).load() df.select(df("rowkey").as("rowkey"), df("`personal.first.name`").as("firstname")).show(10) HBase table=personal_data Phoenix view=personal_data column family=personal column name=first.name – AnumNuma Feb 21 '21 at 19:18
  • val df = spark.sqlContext.read.format("org.apache.phoenix.spark").option("table", "\"personal_data\"").option("zkUrl", zkurl).load() df.select(df("rowkey").as("rowkey"), df("`personal.first.name`").as("firstname")).show(10) – AnumNuma Feb 21 '21 at 19:19

0 Answers0