Writing GZIP file with nio

Question

This is my code:

WritableByteChannel channel = null;
GZIPOutputStream out = null;
try {
     channel = Channels.newChannel(new FileOutputStream("C:\\temp\\111.zip"));

     out = new GZIPOutputStream(Channels.newOutputStream(channel));
    for (long i = 0; i < 10000000000; i++) {    
       out.write(("string" + i + "\n").getBytes());
     } 

   } catch (Exception e) {
        e.printStackTrace();
    } finally {
        try {
            if (channel != null) {
                channel.close();
            }
        } catch (Exception e) {
        }
        try {
            if (out != null) {
                out.close();
            }
        } catch (Exception e) {
        }
    }    }

I get zip but it's contents is damaged.

score 2 · Answer 1 · answered Feb 22 '12 at 11:49

2

I would do:

GZIPOutputStream gzipOS = new GZIPOutputStream(new FileOutputStream("C:\\temp\\111.gz"));
WritableByteChannel out = Channels.newChannel(gzipOS);

And simply use out.write() to write using NIO. Don't forget to close resources later.

answered Feb 22 '12 at 11:49

Buhake Sindi

87,898
29
167
228

out.write requires ByteBuffer – omrid Feb 22 '12 at 11:53
anyway out.write(ByteBuffer.wrap(("string" + "\n").getBytes())); same result damaged file – omrid Feb 22 '12 at 11:58
Yes, you will need `ByteBuffer`. And what you do mean by damaged file? – Buhake Sindi Feb 22 '12 at 12:03
well, the zip file has size bu the file inside has size of 0. when i try to view it with some 3rd party software i get error – omrid Feb 22 '12 at 12:04
I'm assuming you didn't `flush()` and `close()` `gzipOS`. That's only when the data is written out. – Buhake Sindi Feb 22 '12 at 13:29

Tom van der Woerdt · Accepted Answer · 2012-02-22T11:51:27.147

1

Why are you saving it as zip if you're using a gzip stream? Use .gz as the extension.

Edit
Assuming that it's not the .zip extension at fault here (it's still bad though):

You should probably consider calling out.finish() before closing it.
I'm pretty sure you don't need all the channel stuff. You can simply pass the FileOutputStream to GZIPOutputStream

edited Feb 22 '12 at 11:51

answered Feb 22 '12 at 11:45

Tom van der Woerdt

29,532
7
72
105

I want to use nio to improve performance but finish wroked!!! I am runing the test for a huge file now... – omrid Feb 22 '12 at 11:59
aother question: how to set name for zipped file? right now it has the same name of the zip but without extension. – omrid Feb 22 '12 at 12:06
@omrid where does it say that NIO will significantly improve the performance? Have you measured it? Do you have a performance problem? – user207421 Feb 23 '12 at 09:40
@EJP NIO won't significantly improve performance in this case, because it's GZIP that does the actual hard work. – Tom van der Woerdt Feb 23 '12 at 12:35
@TomvanderWoerdt Exactly my point. The overhead lies elsewhere. – user207421 Feb 24 '12 at 08:55
At best the use of NIO is a wash here. At worst, it will actually slow it down. The problem is that when you mix IO (GZipOutputStream) with NIO (WritableByteChannel), you have simply created an additional layer of translation (from byte[] to ByteBuffer) and method calls (OutputStream.write(byte[], int, int) call through to WritableByteChannel.write(ByteBuffer)). – Brett Okken Oct 17 '13 at 13:40

score 1 · Answer 3 · answered Feb 22 '12 at 12:38

When you call out.close() it will close the underlying stream/channel as well.

If you close the underlying channel first any buffered data or footer cannot be written.

The GZIP format contains a CRC32 which must be at the end, cannot be written until you clsoe the stream and I expect this is missing so the file contents cannot be validated. The simplest solution is to not close the underlying channel yourself.

BTW: As a rule it is usually best to close resources in the reverse order they were created. ;)

Writing GZIP file with nio

3 Answers3

Linked