ERROR: NotImplementedException: Scan of table in format 'TEXT' is not supported because the table has a column with a complex type 'ARRAY'

Question

I an new with BigData technologies. I have created one table with column datatype array.

CREATE TABLE movies (
    movie_id int, 
    title string, 
    genres ARRAY<STRING>
) 
ROW FORMAT DELIMITED 
FIELDS TERMINATED BY ':' 
COLLECTION ITEMS TERMINATED BY '|' 
MAP KEYS TERMINATED BY '#' 
LINES TERMINATED BY '\n';

And loaded some data into table. Now I am running a select query then it is showing below error in Impala.

SELECT COUNT(*) AS total_movies FROM movies;

ERROR: NotImplementedException: Scan of table 'assignment_hive_impala.movies' in format 'TEXT' is not supported because the table has a column 'genres' with a complex type 'ARRAY<STRING>'.
Complex types are supported for these file formats: PARQUET.

I am not sure why it is showing this error. Can anyone please explain for error and help me to resolve it?

Thank you

score 0 · Answer 1 · answered Nov 01 '19 at 19:32

It looks like you're loading (or your source file/data) data from text file. As noted in error you can't use text files with complex types.

If you really need to test with complex type convert your source data to Parquet and load data

You can follow this link to convert a file (here it's csv) to parquet.

ERROR: NotImplementedException: Scan of table in format 'TEXT' is not supported because the table has a column with a complex type 'ARRAY'

1 Answers1