Convert Avro To Parquet in Apache Nifi

Question

I wanted to convert my json data to parquet form.

I have used GenerateFlowFile and put dummy json value [{"firstname":"prathik","age":21},{"firstname":"arun","age":22}].

I have already converted json to Avro using ConvertRecord processor, i have then used ConvertAvroToParquet processor.

I got the following output (When viewed as original):

PAR1&&,6(prathikarunprathikarun,(<Hnifi.person% firstname%%age,&% firstnamenn&<6(prathikarun&v%agejj&v<(Ø,parquet.avro.schema€{"type":"record","name":"person","namespace":"nifi","fields":[{"name":"firstname","type":"string"},{"name":"age","type":"int"}]}writer.model.nameavroJparquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a),ÃPAR1

Where is a red dot.

I tried viewing the output in "formatted" form but i get this message : Unable to generate view of data: Not a data file

Thanks in advance

You won't be able to view Parquet in the content viewer in NiFi. You can send the flow file to PutFile and then use parquet-tools — Bryan Bende, Aug 25 '20 at 12:59
https://github.com/apache/parquet-mr/tree/master/parquet-tools https://stackoverflow.com/questions/53306327/installing-parquet-tools — Bryan Bende, Aug 25 '20 at 13:38

score 0 · Answer 1 · answered Nov 20 '20 at 17:45

Parquet is a binary file format and not intended to be viewable within the NiFi content viewer. In newer versions of NiFi you have the ParquetWriter than can be used with ConvertRecord to do the conversion and get the converted Avro data ready to be uploaded.

Convert Avro To Parquet in Apache Nifi

1 Answers1