Error while reading Snappy compressed file

Question

I am reading a Snappy Compressed file from local through java.

File snappyFile = new File(fileName);  
Configuration conf = new Configuration();               
CompressionCodec codec = (CompressionCodec)
ReflectionUtils.newInstance(SnappyCodec.class, conf);
FileInputStream is2 = new FileInputStream(snappyFile);
CompressionInputStream cis = codec.createInputStream(is2);
BufferedReader cr = new BufferedReader(new InputStreamReader(cis));

My code is getting exception at ReflectionUtils.newInstance

Exception in thread "main" java.lang.RuntimeException: java.lang.NoSuchMethodException: org.xerial.snappy.SnappyCodec.<init>()
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
    at org.finra.oats.AWS.AWSnappyRead.run(AWSnappyRead.java:64)
    at org.finra.oats.AWS.AWSnappyRead.main(AWSnappyRead.java:48)
Caused by: java.lang.NoSuchMethodException: org.xerial.snappy.SnappyCodec.<init>()
    at java.lang.Class.getConstructor0(Class.java:2849)
    at java.lang.Class.getDeclaredConstructor(Class.java:2053)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
    ... 2 more

Is it because of version mismatch problem. I am using snappy 1.1.1.2 version

From your question and the provided statcktrace it's not really clear what you want to achieve. Do you want to read a Snappy compressed file from the filesystem? — SubOptimal, Apr 23 '15 at 11:31
You should rephrase your question. Seems you try to read a Hadoop sequence file. Maybe this SO thread [Reading and Writing Sequencefile using Hadoop 2.0 Apis](http://stackoverflow.com/questions/16070587) could help you. — SubOptimal, Apr 23 '15 at 12:15
I had already posted it there [link](http://stackoverflow.com/questions/29816067/how-to-read-snappy-compressed-file-from-s3-in-java). you can refer to the link just to get a brief idea. — salmanbw, Apr 23 '15 at 12:41

SubOptimal · Accepted Answer · 2015-04-23T12:22:12.340

1

To read and write a Snappy compressed file you could use the provided SnappyInputStream / SnappyOutputStream classes.

String fileName = "foo.snap";

// write a snappy compressed file
try (OutputStream os = new SnappyOutputStream(new FileOutputStream(fileName))) {
    os.write("Hello Snappy-World".getBytes(Charset.defaultCharset()));
}

// read a snappy compressed file
try (InputStream is = new SnappyInputStream(new FileInputStream(fileName))) {
    byte[] bytes = new byte[100];
    is.read(bytes);
    System.out.println(new String(bytes, Charset.defaultCharset()));
}

// check if the file is compressed with the snappy algorithm
try (InputStream is = new FileInputStream(fileName)) {
    SnappyCodec readHeader = SnappyCodec.readHeader(is);
    if (readHeader.isValidMagicHeader()) {
        System.out.println("is a Snappy compressed file");
        System.out.printf("%s: %d%n%s: %d%n", 
                "compatible version", readHeader.compatibleVersion,
                "version", readHeader.version
        );
    } else {
        System.out.println("is not a Snappy compressed file");                
    }
}

edited Apr 23 '15 at 12:22

answered Apr 23 '15 at 11:30

SubOptimal

22,518
3
53
69

yes SubOptimal, i want to read a Snappy compressed file from the filesystem. I tried your code but i m getting this error. java.io.IOException: FAILED_TO_UNCOMPRESS(5) at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:84) – salmanbw Apr 23 '15 at 11:40
@user3375762 Could be differen reasons. The file is not compressed with Snappy or is corrupted. Can you sent the hexdump of the first 16 bytes? – SubOptimal Apr 23 '15 at 11:51
@user3375762 This looks not like a snappy compressed file. The magic header of a snappy compressed file is defined as `MAGIC_HEADER = new byte[] { -126, 'S', 'N', 'A', 'P', 'P', 'Y', 0 };` – SubOptimal Apr 23 '15 at 12:11
@user3375762 Im not familar with hadoop. So rephrase your question to be clear what you want to do and add the hadoop tag. I'm sure there reader which like to help you on this topic. – SubOptimal Apr 23 '15 at 12:24

Error while reading Snappy compressed file

1 Answers1