16

I got some pig generated files with part-r-00000.deflate extension. I know this is a compressed file. How do I generate a normal file in a readable format. When I used hadoop fs -text, I cannot get plaintext output. The output is still binary. How can I fix this problem?

xgMz
  • 3,334
  • 2
  • 30
  • 23
Himateja Madala
  • 321
  • 1
  • 4
  • 16

2 Answers2

20

You might be using a quite old Hadoop version (e.g: 0.20.0) in which fs -text can't inflate the compressed file.

As a workaround you may try this one-liner (based on this answer):

hadoop fs -text file.deflate | perl -MCompress::Zlib -e 'undef $/; print uncompress(<>)'
Community
  • 1
  • 1
Lorand Bendig
  • 10,630
  • 1
  • 38
  • 45
17

you can decompress on the fly by using this command

hdfs dfs -text file.deflate | hdfs dfs -put - uncompressed_destination_file

guignol
  • 404
  • 4
  • 10