How to extract data from corrupted gzip files in python

Question

I'm using the python gzip library to expand files, and a few of them are corrupted. The exact error is this:

Error -3 while decompressing: invalid block type

Is it possible to either read all the data before the broken point of the file, or to somehow skip over the broken point and read what's before and after? The compressed files are basically lines of text, and I would like to recover as much data as possible.

Thanks

You can try to use `os.system('gunzip < corrupted.gz > out')`, it should extract data before bad sector. But extract data after bad sector - I don't now any reliable approach. — serkos, Nov 07 '14 at 05:24
The author of `gzip` [has this to say](http://www.gzip.org/recover.txt). Recovering data before the damage is done as @serkos suggests above. Recovering data after looks hard, and the suggested method involves editing the `c` source of gzip, which is not the Python solution you are looking for. — Tony, Nov 07 '14 at 07:00

score 4 · Answer 1 · answered Nov 07 '14 at 16:08

Hopefully someone finds this useful:

# http://stackoverflow.com/questions/2423866/python-decompressing-gzip-chunk-by-chunk
# http://stackoverflow.com/questions/3122145/zlib-error-error-3-while-decompressing-incorrect-header-check/22310760
def read_corrupted_file(filename, CHUNKSIZE=1024):
    d = zlib.decompressobj(zlib.MAX_WBITS | 32)
    with open(filename, 'rb') as f:
        result_str = ''
        buffer=f.read(CHUNKSIZE)
        try:
            while buffer:
                result_str += d.decompress(buffer)
                buffer=f.read(CHUNKSIZE)
        except Exception as e:
            print 'Error: %s -> %s' % (filename, e.message)
        return result_str

score 2 · Answer 2 · answered Nov 07 '14 at 16:06

You can use the Python zlib interface to decompress a piece at a time, which will give you the decompressed data up to the bad block. Note that the corruption likely precedes the point where it is caught, so some amount at the end of the decompressed data you get may be corrupted.

Recovering data after the error is pretty much impossible (see link in comment to question), unless the gzip file was specially prepared to have recovery points. The gzip utility itself doesn't do that.

How to extract data from corrupted gzip files in python

2 Answers2