Error decompressing gzip data using Qt

Question

I'm decompressing gzip data received from a http server, using the zlib library from Qt. Because qUncompress was no good, I followed the advice given here: Qt quncompress gzip data and created my own method to uncompress the gzip data, like this:

 QByteArray gzipDecompress( QByteArray compressData )
 {
    //strip header
    compressData.remove(0, 10);

    const int buffer_size = 16384;
    quint8 buffer[buffer_size];

    z_stream cmpr_stream;
    cmpr_stream.next_in = (unsigned char *)compressData.data();
    cmpr_stream.avail_in = compressData.size();
    cmpr_stream.total_in = 0;

    cmpr_stream.next_out = buffer;
    cmpr_stream.avail_out = buffer_size;
    cmpr_stream.total_out = 0;

    cmpr_stream.zalloc = Z_NULL;
    cmpr_stream.zfree = Z_NULL;
    cmpr_stream.opaque = Z_NULL;

    int status = inflateInit2( &cmpr_stream, -8 );
    if (status != Z_OK) {
        qDebug() << "cmpr_stream error!";
    }

    QByteArray uncompressed;
    do {
        cmpr_stream.next_out = buffer;
        cmpr_stream.avail_out = buffer_size;

        status = inflate( &cmpr_stream, Z_NO_FLUSH );

        if (status == Z_OK || status == Z_STREAM_END)
        {
            QByteArray chunk = QByteArray::fromRawData((char *)buffer, buffer_size - cmpr_stream.avail_out);
            uncompressed.append( chunk );
        }
        else
        {
            inflateEnd(&cmpr_stream);
            break;
        }

        if (status == Z_STREAM_END)
        {
            inflateEnd(&cmpr_stream);
            break;
        }
    }
    while (cmpr_stream.avail_out == 0);

    return uncompressed;
 }

Eveything seems to work fine if the decompressed data fits into the output buffer (ie. is smaller than 16 Kb). If it doesn't, the second call to inflate returns a Z_DATA_ERROR. I know for sure the data is correct because the same chunk of data is correctly decompressed if the output buffer is made large enough.

The server doesn't return a header with the size of the uncompressed data (only the size of the compressed one) so I followed the usage instructions in zlib: http://www.zlib.net/zlib_how.html

And they do exactly what I'm doing. Any idea what I could be missing? the next_in and avail_in members in the stream seem to be updated correctly after the first iteration. Oh, and if it's any useful, the error message when the data error is issued is: "invalid distance too far back".

Any thoughts? Thanks.

score 4 · Accepted Answer · answered Apr 21 '11 at 12:10

4

The Deflate/Inflate compression/decompression algorithm uses a 32Kb circular buffer. So a 16Kb buffer can never work if the decompressed data is bigger than 16Kb. (Not strictly true, because the data is allowed to be split into blocks, but you need to assume that there may be 32Kb blocks in there.) So just set buffer_size = 32768 and you should be OK.

answered Apr 21 '11 at 12:10

TonyK

16,761
4
37
72

I have the same problem and I solved this by using the folowing method Try this – pranavjayadev Nov 30 '12 at 10:41

Error decompressing gzip data using Qt

1 Answers1