Streaming contents from lz4 file using InputStreamReader - Stream Corrupted - Java

Question

I am trying to stream data from the lz4 compressed file and write it to StringBuilder. But I am getting Stream is corrupted exception. Below is my code.

    import java.io.ByteArrayInputStream;
    import java.io.File;
    import java.io.IOException;
    import java.io.InputStreamReader;
    import java.nio.charset.StandardCharsets;
    import net.jpountz.lz4.LZ4BlockInputStream;
    import org.apache.commons.io.FileUtils;
    
          public class Lz4TestRead {
        
            public static void main(String[] args) throws IOException {
                byte[] data = FileUtils.readFileToByteArray(new File("D:\\sparker\\input\\payload.lz4"));
                try(ByteArrayInputStream bis = new ByteArrayInputStream(data);
                    LZ4BlockInputStream lz4BlockInputStream = new LZ4BlockInputStream(bis)) {
                    InputStreamReader streamReader = new InputStreamReader(lz4BlockInputStream, StandardCharsets.UTF_8);
                    StringBuilder sb = new StringBuilder();
                    while (true) {
                        int read = streamReader.read();
                        if (read < 0) {
                            break;
                        }
                        sb.append((char) read);
                    }
                    streamReader.close();
                    System.out.println(sb.toString());
        
                }catch (Exception ex) {
                    ex.printStackTrace();
                }
        
        
            }
        }

I am getting below exception

java.io.IOException: Stream is corrupted
    at net.jpountz.lz4.LZ4BlockInputStream.refill(LZ4BlockInputStream.java:202)
    at net.jpountz.lz4.LZ4BlockInputStream.read(LZ4BlockInputStream.java:157)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
    at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:127)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:112)
    at java.io.InputStreamReader.read(InputStreamReader.java:168)
    at com.test.Lz4TestRead.main(Lz4TestRead.java:21)

Can anyone help me what I am doing wrong.

Maybe this post helps you https://stackoverflow.com/questions/36012183/java-lz4-compression-using-input-output-streams — Octavian R., Dec 02 '20 at 07:56

Octavian R. · Accepted Answer · 2020-12-02T08:31:44.743

This code works fine

UPDATE: updated the code to respect the first comment

import net.jpountz.lz4.LZ4BlockInputStream;
import net.jpountz.lz4.LZ4BlockOutputStream;

import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;

public class Lz4TestRead {

    public static void main(String[] args) throws IOException {

        //generate an LZ4 encrypted file
        byte[] buf = new byte[1024];
        try (LZ4BlockOutputStream out = new LZ4BlockOutputStream(new FileOutputStream("/tmp/text.lz4"), 1024);
             ByteArrayInputStream in = new ByteArrayInputStream("hello world this is a LZ4 compressed text".getBytes())) {
            int len;
            while ((len = in.read(buf)) > 0) {
                out.write(buf, 0, len);
            }
        }

        StringBuilder sb = new StringBuilder();
        try (ByteArrayInputStream bis = new ByteArrayInputStream(Files.readAllBytes(Path.of("/tmp/text.lz4")));
             LZ4BlockInputStream lz4BlockInputStream = new LZ4BlockInputStream(bis);
             InputStreamReader streamReader = new InputStreamReader(lz4BlockInputStream, StandardCharsets.UTF_8)) {
            while (true) {
                int read = streamReader.read();
                if (read < 0) {
                    break;
                }
                sb.append((char) read);
            }
        }

        System.out.println(sb.toString());

    }
}

But keep in mind that you have to compress the data using LZ4BlockOutputStream in order to be able to decompress it using LZ4BlockInputStream

For me, I have to use ByteArrayInputStream. So I read file as BytearrayInputStream and trying to read it. it still throws same exception at lz4BlockInputStream.read(buf). And also I used the proper lz4 file. And also I have a requirement of reading character by character. Thats why I used InpuytStreamReader. — user2731629, Dec 02 '20 at 08:19
Looks like then issue with my lz4 file. Thanks for your help. — user2731629, Dec 02 '20 at 08:44

Streaming contents from lz4 file using InputStreamReader - Stream Corrupted - Java

1 Answers1