2

We have Spark 2.1, Thrift JDBC/ODBC server configured with Hive 2.1.1 and using Beeline.

DDL statements like

CREATE TABLE, CREATE TABLE LIKE, CREATE TABLE LIKE, ALTER TABLE SET TBLPROPERTIES

works well.

But ALTER TABLE ADD COLUMNS fails

create table tbl1 (id int, c1 string, c2 string);
alter table tbl1 add columns (c3 string);

Error: org.apache.spark.sql.catalyst.parser.ParseException: Operation not allowed: alter table add columns(line 1, pos 0)

== SQL == alter table tbl1 add columns (c3 string) ^^^ (state=,code=0)

Any hint would be appreciated

Johann Voppichler
  • 21
  • 1
  • 4

3 Answers3

2

Looks like this is work in progress for Spark v2.x

https://issues.apache.org/jira/browse/SPARK-19261

bejito
  • 43
  • 7
0

What worked for me, hence I'm adding it as a possible answer here, was to use hive directly from the console, all ALTER TABLE statements worked like a charm (we are on Spark 2.0.2 with not much rush to upgrade...).

m-dz
  • 2,342
  • 17
  • 29
-1

Try using HiveContext instead of spark Session. As some of the hive metastore queries does on work well with spark Session.

Ashish Singh
  • 523
  • 3
  • 14