0

I'm currently using sparklyr. I'm able to connect to our cluster but it's defaulting to Hive instead of Impala. Is there an option to default the schema to another schema other than default?

RLW
  • 66
  • 5
  • Possible duplicate of [Access table in other than default scheme (database) from sparklyr](https://stackoverflow.com/questions/43806426/access-table-in-other-than-default-scheme-database-from-sparklyr) –  Sep 13 '18 at 13:34
  • It connects using this method. But when I try to execute a view that has a function like strleft, split_part, or isnull it errors out and doesnt run. Its seems like it's still running from Hive instead of Impala. Does Spark works with Impala? – RLW Sep 13 '18 at 17:57
  • 1
    Spark uses Hive only as a metastore, not as an execution engine. Even if you connect to Impala metastore, all processing will be performed by Spark, not Impala. So in this sense Spark doesn't work with Impala. – zero323 Sep 17 '18 at 14:54

0 Answers0