0

Imported single table using sqoop

...
--as-avrodatafile
--compression-codec ...SnappyCodec
...

Viewing in Hue file browser confirms that it is compressed by displaying a message Output rendered from compressed avro file.

Created external table in Hive as

create external table customers_avro
stored as avro
location '.../path/'
tblproperties('avro.schema.file'='/path/to/customers.avsc')

However, when querying with describe extended tblname shows compressed:false

...
inputFormat:org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.serde2.avro.AvroSerDe, parameters:{serialization.format=1})...

Is there any other property that needs to be set with tblproperties?

Bala
  • 11,068
  • 19
  • 67
  • 120

1 Answers1

0

I would suggest verify if your data is properly compressed. You can use this.

Compression codec detection in Hadoop from the command line,

Also, should be useful know your hive and sqoop versions, try to import from sqoop like this 

--compression-codec snappy
hlagos
  • 7,690
  • 3
  • 23
  • 41