Convert zipped []byte to unzip []byte golang code

Question

I have []byte of zip file. I have to unzip it without creating a new file, and get a []byte of that unzipped file. Please help me to do that.

I am making an API call and the response I get is the []byte in zipped format - I am trying to unzip it - and use it's content for creating a new zip file. So unzip - rezip.

Language: Golang

Code I've used:

func UnzipBytes(zippedBytes []byte) ([]byte, error) {

    reader := bytes.NewReader(zippedBytes)
    zipReader, err := zlib.NewReader(reader)
    if err != nil {
        return nil, err
    }
    defer zipReader.Close()
    p, err := ioutil.ReadAll(zipReader)
    if err != nil {
        return nil, err
    }
    return p, nil
}

I get an error saying "zlib: invalid header"

The code that was initially used to zip the []byte

buffer := new(bytes.Buffer)
zipWriter := zip.NewWriter(buffer)
zipFile, err := zipWriter.Create(file.name)
_, err = zipFile.Write(file.content)

Hex dump of the []byte - the zippedBytes

00059350  78 b4 5b 0d 2b 81 c2 87  35 76 1b 11 4a ec 07 d1  |x.[.+...5v..J...|
00059360  76 77 a2 e1 3b d9 12 e2  51 d4 c5 bd 4b 2f 09 da  |vw..;...Q...K/..|
00059370  f7 21 c7 26 73 1f 8e da  f0 ff a3 52 f6 e2 00 e6  |.!.&s......R....|

Check out the `compress/zlib` package. You can create a `Reader`, providing an input that reads from your `[]byte`, and you can read the decompressed data, obtaining it as a `[]byte` e.g. using `io.ReadAll()`. — icza, Sep 14 '22 at 13:16
(or `archive/zip`, `compress/gzip`, `compress/flate` etc. depending on what you mean by "zipped []byte") — JimB, Sep 14 '22 at 13:16
I've tried that icza it did not work - and I found the same question in stackoverflow for Java - and I just copied and pasted for GoLang - cause it's something that's required and should be fairly straightforward to do. I'm looking for the actual code not advice. Thanks anyways though. — Bouramas, Sep 14 '22 at 13:17
The question itself is under-specified, and "please write code for me" is not an acceptable type of question here (though perhaps it was more acceptable in the past with whatever java question you used as a reference) — JimB, Sep 14 '22 at 13:20
JimB - if you can't be helpful than just please don't help - you can waste your time elsewhere. I am a stackoverflow user for a long time - and what I'm asking is completely straight forward. Since the same exact questions exists and is upvoted in another language - > https://stackoverflow.com/questions/9209450/convert-zip-byte-to-unzip-byte — Bouramas, Sep 14 '22 at 13:23
If the obvious use of the `zip` package didn't work, it would really help to have a [mre] showing _what_ didn't work. Otherwise we're just calling `zip.NewReader` for you. — JimB, Sep 14 '22 at 13:24
If you get _"zlib: invalid header"_, that means your `[]byte` is not a "simple" zlib block. Try the other packages suggested by @JimB. — icza, Sep 14 '22 at 13:33
If you don't know what it is, post the `hex.Dump()` output of its first like 100 bytes and we might be able to help you find out what format it is. — icza, Sep 14 '22 at 13:34
If `zlib` doesn't work for your "zip", you probably either have a complete archive, in which case you can use the example in the [`archive/zip`](https://pkg.go.dev/archive/zip#example-Reader) docs to unpack the files; or you have raw DEFLATE stream, in which case you can use `compress/flate`. — JimB, Sep 14 '22 at 13:45
You used `zip.Writer` to compress the data. You must close it by calling its `Close()` method. And you must use `zip.Reader` to read it, and use `Reader.Open()` with the same name you used when compressed it `file.content`. If you don't know the name when decompressing, you may see all files in the `Reader.Files` header field. — icza, Sep 14 '22 at 13:57
@icza - I've added a small portion of the hex.Dump() - JimB- I am not sure I understand the raw DEFLATE stream you are referring to. I can't create the method with the examples provided. Thanks for trying to help both of you. — Bouramas, Sep 14 '22 at 13:58
don't worry about DEFLATE, it's clear now you are not using that. — JimB, Sep 14 '22 at 14:00
I don't have a file - I have just the []byte at the method shown - so what do you mean you must use Reader.Open() - I've got nothing to open - I've got the contents there in the []byte. that's it. I want to unzip/convert just the []bytes as they are — Bouramas, Sep 14 '22 at 14:01
OK, now the question contains enough info to answer it. See my answer below. — icza, Sep 14 '22 at 14:08

icza · Accepted Answer · 2022-09-14T14:14:02.670

You used zip.Writer to compress the data. You must close it by calling its Writer.Close() method. And you must use zip.Reader to read it, and use Reader.Open() with the same name you used when compressed it (file.name).

This is how it could look like:

func UnzipBytes(name string, zippedBytes []byte) ([]byte, error) {
    reader := bytes.NewReader(zippedBytes)
    zipReader, err := zip.NewReader(reader, int64(len(zippedBytes)))
    if err != nil {
        return nil, err
    }
    f, err := zipReader.Open(name)
    if err != nil {
        panic(err)
    }
    p, err := ioutil.ReadAll(f)
    if err != nil {
        return nil, err
    }
    return p, nil
}

Testing it:

filename := "test.txt"
filecontent := []byte("line1\nline2")

buffer := new(bytes.Buffer)
zipWriter := zip.NewWriter(buffer)
zipFile, err := zipWriter.Create(filename)
if err != nil {
    panic(err)
}
if _, err = zipFile.Write(filecontent); err != nil {
    panic(err)
}
if err = zipWriter.Close(); err != nil {
    panic(err)
}

decoded, err := UnzipBytes(filename, buffer.Bytes())
fmt.Println(err)
fmt.Println(string(decoded))

This will output (try it on the Go Playground):

<nil>
line1
line2

If you don't know the name when decompressing, you may see all files in the Reader.Files header field. You may choose to open the first file:

func UnzipBytes(zippedBytes []byte) ([]byte, error) {
    reader := bytes.NewReader(zippedBytes)
    zipReader, err := zip.NewReader(reader, int64(len(zippedBytes)))
    if err != nil {
        return nil, err
    }
    if len(zipReader.File) == 0 {
        return nil, nil // No file to open / extract
    }
    f, err := zipReader.File[0].Open()
    if err != nil {
        panic(err)
    }
    p, err := ioutil.ReadAll(f)
    if err != nil {
        return nil, err
    }
    return p, nil
}

This outputs the same. Try this one on the Go Playground.

The last function is what I was looking for. I was missing this zipReader.File[0].Open() - Thanks @icza you are a legend! I really appreciate your time. I think this question will also be valuable to others as well - time will tell. Thanks again. — Bouramas, Sep 14 '22 at 14:16

Convert zipped []byte to unzip []byte golang code

1 Answers1

Linked