Table not found error while accessing Hive table using SQLContext of Spark v 1.6

Question

I am getting "Table no found" error while using

SQLContext sqlCon=SQLContext.getOrCreate(saprkContext.sc());

The stack trace is as following :

18/08/16 19:58:31 INFO cluster.YarnClientSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
Exception in thread "main" org.apache.spark.sql.AnalysisException: Table not found: `schema`.`tableName`;
        at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
        at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:54)
        at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:50)
        at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:121)
        at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$foreachUp$1.apply(TreeNode.scala:120)
        at org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$foreachUp$1.apply(TreeNode.scala:120)
        at scala.collection.immutable.List.foreach(List.scala:318)
        at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:120)
        at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:50)
        at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:44)
        at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:35)
        at org.apache.spark.sql.DataFrame.<init>(DataFrame.scala:133)
        at org.apache.spark.sql.DataFrame$.apply(DataFrame.scala:52)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:829)
        at com.ktk.ccemi.MIProcess$1StaticDataLoader.loadData(MIProcess.java:211)
        at com.ktk.ccemi.MIProcess$1StaticDataLoader.<init>(MIProcess.java:167)
        at com.ktk.ccemi.MIProcess.createContext(MIProcess.java:234)
        at com.ktk.ccemi.MIProcess$1.call(MIProcess.java:74)
        at com.ktk.ccemi.MIProcess$1.call(MIProcess.java:1)
        at org.apache.spark.streaming.api.java.JavaStreamingContext$$anonfun$10.apply(JavaStreamingContext.scala:776)

I am using Spark streaming with checkpoint and my version of Spark is 1.6 .

[What is the difference between Apache Spark SQLContext vs HiveContext?](https://stackoverflow.com/q/33666545/8371915) — Alper t. Turker, Aug 16 '18 at 15:06

score 0 · Answer 1 · answered Aug 16 '18 at 15:01

0

To access Hive tables you need:

HiveContext in Spark 1.x
SparkSession with Hive support enabled in Spark 2.x.

Accessing Hive tables through basic SQLContext is not supported.

answered Aug 16 '18 at 15:01

user10235084

1

Thats not true, quoting from Spark Documentation "With a SQLContext, applications can create DataFrames from an existing RDD, from a Hive table, or from data sources." [ https://spark.apache.org/docs/1.6.0/sql-programming-guide.html#creating-dataframes ] (https://spark.apache.org/docs/1.6.0/sql-programming-guide.html#creating-dataframes ) – sanjeev kumar Aug 16 '18 at 18:14

Table not found error while accessing Hive table using SQLContext of Spark v 1.6

1 Answers1