Unable to inflate a buffer deflated with java.util.zip.Deflater using Apache MINA compression filter

Question

This test:

import java.util.zip.Deflater;
import org.apache.mina.core.buffer.IoBuffer;
import org.apache.mina.filter.support.Zlib;
import org.junit.*;

public class ZlibTest {
    private Deflater deflater = null;

    private Zlib inflater = null;

    @Before
    public void setUp() throws Exception {
        deflater = new Deflater(Deflater.BEST_COMPRESSION);
        deflater.setStrategy(Deflater.DEFAULT_STRATEGY);
        inflater = new Zlib(Zlib.COMPRESSION_MAX, Zlib.MODE_INFLATER);
    }

    @Test
    public void testInflate() throws Exception {
        byte[] compressed = new byte[14];
        deflater.setInput(new byte[] {1});
        deflater.finish();
        int bytesCompressed = deflater.deflate(compressed);
        IoBuffer compressedBuffer = IoBuffer.wrap(compressed, 0, bytesCompressed);
        System.out.println(compressedBuffer);

        IoBuffer byteUncompressed = inflater.inflate(compressedBuffer);
    }
}

fails:

java.io.IOException: Unknown error. Error code : 1
    at org.apache.mina.filter.support.Zlib.inflate(Zlib.java:136)
    at ZlibTest.testInflate(ZlibTest.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
    at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:46)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

What is going wrong here?

UPDATE: if I add

        case JZlib.Z_STREAM_END:

to line 139 in Zlib.java, it decodes fine.

Answer 1

With regards to http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4255743 (famous, 13 year old bug - would go to the highschool shortly if it were my kid), the fix has been delivered in Java 7 (at last); indeed, there's a

public int deflate(byte[] b, int off, int len, int flush) {
    if (b == null) {
        throw new NullPointerException();
    }
    if (off < 0 || len < 0 || off > b.length - len) {
        throw new ArrayIndexOutOfBoundsException();
    }
    synchronized (zsRef) {
        ensureOpen();
        if (flush == NO_FLUSH || flush == SYNC_FLUSH ||
            flush == FULL_FLUSH)
            return deflateBytes(zsRef.address(), b, off, len, flush);
        throw new IllegalArgumentException();
    }
}

section of code added to Deflater class. So today (2012), you can just stick with java.util if that functionality is all that you need.

Answer 2

You've hit the 'design choice' of the Java Deflater class that caused the creation of JZlib - you can't control the FLUSH parameter for zlib.

The output from the Deflater class for your one-byte array of {1}:

[0] 120 
[1] -38 
[2] 99  
[3] 4   
[4] 0   
[5] 0   <<
[6] 2   <<
[7] 0   <<
[8] 2   <<

The output from the Zlib class when deflating for your one-byte array of {1}:

[0] 120 
[1] -38 
[2] 98  
[3] 4   
[4] 0   
[5] 0   <<
[6] 0   <<
[7] -1  <<
[8] -1  <<

From Zlib manual

If the parameter flush is set to Z_SYNC_FLUSH, all pending output is flushed to the output buffer and the output is aligned on a byte boundary, so that the decompressor can get all input data available so far. (In particular avail_in is zero after the call if enough output space has been provided before the call.) Flushing may degrade compression for some compression algorithms and so it should be used only when necessary. This completes the current deflate block and follows it with an empty stored block that is three bits plus filler bits to the next byte, followed by four bytes (00 00 ff ff).

From JZlib - Why JZlib?

Java Platform API provides packages 'java.util.zip.*' for accessing to zlib, but that support is very limited if you need to use the essence of zlib. For example, we needed to full access to zlib to add the packet compression support to pure Java SSH system, but they are useless for our requirements. ...

To implement this functionality, the Z_PARTIAL_FLUSH mode of zlib must be used, however JDK does not permit us to do so. It seems that this problem has been well known and some people have already reported to JavaSoft's BugParade(for example, BugId:4255743), but any positive response has not been returned from JavaSoft, so this problem will not be solved forever. This is our motivation to hack JZlib.

Answer 3

I wouldn't assume that Zlib and Inflater/deflater use the same data protocol for sending data. They may use the same underlying compression but I suspect Zlib is expecting the stream to contain information for its own used, not just raw data.

---

Deflater deflater = new Deflater(Deflater.BEST_COMPRESSION);
deflater.setStrategy(Deflater.DEFAULT_STRATEGY);

byte[] compressed = new byte[10];
deflater.setInput(new byte[]{1});
deflater.finish();
int bytesCompressed = deflater.deflate(compressed);
System.out.println("bytesCompressed=" + bytesCompressed + " " + Arrays.toString(compressed));

Inflater inflater = new Inflater();
inflater.setInput(compressed, 0, bytesCompressed);
byte[] decompressed = new byte[2];
int byteDecompressed = inflater.inflate(decompressed);

System.out.println("bytesInflated=" + byteDecompressed + " " + Arrays.toString(decompressed));

prints

bytesCompressed=9 [120, -38, 99, 4, 0, 0, 2, 0, 2, 0]
bytesInflated=1 [1, 0]