1

I am using AbInitio and attempting to have my results from my query in my Input Table populated into hdfs. I am wanting the format in parquet. I tried using the dml to hive text but the following is my results and I am not sure what this means.

$ dml-to-hive text $AI_DML/myprojectdml.dml
Usage: dml-to-avro <record_format> <output_file>

   or: dml-to-avro help

  <record-format> is one of:
       <filename>           Read record format from file
       -string <string>     Read record format from string

  <output_file> is one of:
       <filename>           Output Avro schema to file
       -                    Output Avro schema to standard output

I also tried using the Write Hive Table component but I receive the following error:

[B276]
The internal charset "XXcharset_NONE" was encountered when a valid character set data
structure was expected.   One possible cause of this error is that you specified a 
character set to the Co>Operating System that is misspelled or otherwise incorrect.
If you cannot resolve the error please contact Customer Support.

Any help would be great, I am trying to have my output to hdfs in parquet.

Thanks, Chris Richardson

ccr6924
  • 11
  • 1
  • 2

1 Answers1

3

I know this is a late reply, but if you're still working on this or somebody else stumbles onto this like I did, I think I've found a solution.

I used dml-to-hive to create a DML for parquet format and write it to a file.

dml-to-hive parquet current.dml > parquet.dml

Once this dml is created, you can use it on the in port of the "Write HDFS" component. Double click the component, go to Port tab, click Radio button "Use File" and then point it to parquet.dml

Then, just set the WRITE_FORMAT choice to parquet and give it a whirl. I was able to create parquet, orc, and avro files using the above process.

ChicagoCyclist
  • 109
  • 9