Creating Zip file while client is downloading

Question

I try to develop something like dropbox(very basic one). For one file to download, it's really easy. Just use servletoutputstream. what i want is: when client asks me multiple file, i zip files in server side then send to user. But if file is big it takes too many times to zip them and send to user.

is there any way to send files while they are compressing?

thanks for your help.

Answer 1

Part of the Java API for ZIP files is actually desgined to provide "on the fly" compression. It all fits nicely both with the java.io API and the servlet API, which means this is even... kind of easy (no multithreading required - even for performance reason, because usually your CPU will probably be faster at ZIPping than your network will be at sending contents).

The part you'll be interacting with is ZipOutputStream. It is a FilterOutputStream (which means it is designed to wrap an outputstream that already exists - in your case, that would be the respone's OutputStream), and will compress every byte you send it, using ZIP compression.

So, say you have a get request

protected void doGet(HttpServletRequest req, HttpServletResponse resp)
    throws ServletException, IOException {

    // Your code to handle the request
    List<YourFileObject> responseFiles = ... // Whatever you need to do 

    // We declare that the response will contain raw bytes
    response.setContentType("application/octet-stream");

    // We open a ZIP output stream
    try (ZipOutputStream zipStream = new ZipOutputStream(response.getOutputStream()) {// This is Java 7, but not that different from java 6

        // We need to loop over each files you want to send
        for(YourFileObject fileToSend : responseFiles) {
            // We give a name to the file
            zipStream.putNextEntry(new ZipEntry(fileToSend.getName()));
            // and we copy its content
            copy(fileToSend, zipStream);
        }
    }
}

Of course, you should do proper exception handling. A couple quick notes though :

1. The ZIP file format mandates that each file has a name, so you must create a new ZipEntry each time you start a new file (you'll probably get an IllegalStateException if you do not, anyway)
2. Proper use of the API would be that you close each entry once you are done writing to it (at the end of the file). BUT : the Java implementation does that for you : each time you call putNextEntry it closes the previous one (if need be) all by itself
3. Likewise, you must not forget to close the ZIP stream, beacuse, this will properly close the last entry AND flush everything that is needed to create a proper ZIP file. Failure to do so will result in a corrupt file. Here, the try with resources statement does this : it closes the ZipOutputStream once everything is written to it.
4. The copy method here is just what you would use to transfert all the bytes from the original file to the outputstream, there is nothing ZIP specific about it. Just call outputStream.write(byte[] bytes).

**EDIT : ** to clarify...

For example, given a YourFileType that has the following methods :

public interface YourFileType {

    public byte[] getContent();

    public InputStream getContentAsStream();

}

Then the copy method could look like (this is all very basic Java IO, you could maybe use a library such as commons io to not reinvent the wheel...)

public void copy(YourFileType file, OutputStream os) throws IOException {
    os.write(file.getContent());
}

Or, for a full streaming implementation :

public void copy(YourFileType file, OutputStream os) throws IOException {
    try (InputStream fileContent = file.getContentAsStream()) {
        byte[] buffer = new byte[4096]; // 4096 is kind of a magic number
        int readBytesCount = 0;
        while((readBytesCount = fileContent.read(buffer)) >= 0) {
            os.write(buffer, 0, readBytesCount);
        }
    }
}

Using this kind of implementation, your client will start receiveing a response almost as soon as you start writing to the ZIPOutputStream (the only delay would be that of internal buffers), meaning it should not timeout (unless you spent too long buliding the content to send - but that would not be the ZIPping part fault's).