How can libzip be used from C to create a zip file in memory?

Question

I am trying to using libzip to create a zip file directly in memory, not to a file on disk. My code is quite basic at the moment as I am getting stuck on creating the necessary zip_t struct from a zip_source_buffer:

#include <stdio.h>
#include <string.h>
#include <zip.h>

int main(int argc, char *arrv[])
{
    char buffer[65536] = {};
    zip_error_t error;
    zip_source_t *zs = zip_source_buffer_create(buffer, sizeof(buffer), 0, &error);

    int err = zip_source_begin_write(zs);
    printf("%p %d '%s'\n", zs, err, zip_error_strerror(&error)); 
    zip_error_fini(&error);

    zip_t * zip = zip_open_from_source(zs, ZIP_CREATE, &error);
    printf("%p '%s'\n", zip, zip_error_strerror(&error)); 
    zip_error_fini(&error);
}

the code compiles and runs, but thows an error:

$ ./ztest 
0xdd50a0 0 'No error'
(nil) 'Not a zip archive'

It's not clear if the begin_write() is needed, but it doesn't generate an error and I get the same result without it.

What am I missing? Thanks

I think I may have misunderstood the usage here and the zip_source functions are there for the CONTENT of the file, not the ZIP file itself - is this case? can I actually do what I want with libzip, i.e. create a zip file in memory? — Dave Lawrence, Feb 13 '18 at 15:47
It's not clear. I took a quick look at their docs and they're reference rather than how-to. What system are you running on? Why does it need to be in memory? You could do a memory mapped file. — lurker, Feb 13 '18 at 16:01
linux. The file is being built for transmission over a serial data link and does ever need to be created on-disk. and yes, I agree the docs are very much for reference! — Dave Lawrence, Feb 13 '18 at 16:29
The documentation is quite terse, so what's not clear to me is whether you need to do `zip_open_from_source` after you've done `zip_source_begin_write`. Based upon their names, I'd have almost thought the other way around, but just a guess. However, your `zip_source_begin_write` appeared to succeed, so not sure. As an aside, you should check for `zs == NULL` after your `zip_source_buffer_create` call. — lurker, Feb 13 '18 at 16:54
yes, it's very not clear. Once you get to zip_file_add(), you then get to notion that a zip_source_buffer is an INPUT block of data not the destination .ZIP file (hence my first comment)! when (and if) I get this working, I will of course add the rest of the rc/error checks — Dave Lawrence, Feb 13 '18 at 17:02
On the bright side, there are few permutations to try if you want to do the "educated pseudo Monte Carlo" approach. — lurker, Feb 13 '18 at 17:12
I was struggling with the same issue, but [this example](https://github.com/nih-at/libzip/blob/main/examples/in-memory.c) in the repo is very helpful in this direction. Apparently there is no need for `zip_source_begin_write`, you just need to write code that change the zip after `zip_open_from_source` and in the end copy the content of the zip_source_t* into the c++ buffer (a bit counter-intuitive, I know, but it is what it is). — DrSushi, Jul 27 '23 at 11:00

score 3 · Answer 1 · answered Oct 15 '19 at 10:59

My 2 cent if anybody yet has interest in question.

Do not forget to zip_source_keep before zip_open_from_source, as zip_close will free source, but we want to use its content.

int flags = 0;
zip_error_t error;
zip_source_t *zsmem = zip_source_buffer_create(0, 0, 0, &error);
//we can fill up new zip with an old one (some template, for example)
if (exdata){
    zip_source_begin_write(zsmem);
    zip_source_write(zsmem, exdata, exdata_sz); 
    zip_source_commit_write(zsmem); 
}else
    flags |= ZIP_TRUNCATE;

zip_source_keep(zsmem); //!!
struct zip* arc = zip_open_from_source(zsmem, flags, &error);

//do more stuff adding, removing files , comments, etc

zip_close(arc);

zip_source_open(zsmem);
zip_source_seek(zsmem, 0, SEEK_END);
zip_int64_t sz = zip_source_tell(zsmem);
zip_source_seek(zsmem, 0, SEEK_SET);
zip_source_read(zsmem, outbuffer, sz);
zip_source_close(zsmem);

zip_source_free(zsmem);

send_over_network(outbuffer);

score 1 · Answer 2 · answered Jul 04 '19 at 02:35

Use ZIP_TRUNCATE instead of ZIP_CREATE. The archive always exists in the case of a buffer source, so ZIP_CREATE has no effect, but a stream of '0's is not a valid zip file. ZIP_TRUNCATE tells libzip to ignore the buffer's current contents (i.e. override all the '0's with the new archive).

zip_source_begin_write() is called internally by libzip. It's not necessary for it to be called in this kind of situation.

How can libzip be used from C to create a zip file in memory?

2 Answers2