Extract gzip via passive library

Question

I need to extract a large gzip data stream in C++ (on Linux) into memory. I intended to use libarchive, but I have the limit that I cannot block or use any kind of synchronization.

AFAIR in order to use libarchive's custom reader it has to block, till data is available.

So I would like it to have the other way around. A library, which has callbacks for entries/progress and the ability to put data in manually.

Is there any library out there that does this or is it faster to modify libarchive to work the other way around?

@hyde There are lots of reasons not to use threads. Nginx exists because of them. — Alex Chamberlain, Jan 30 '13 at 11:52
I assumed you can't use syncronization which blocks. If you can't use any kind of IPC mechanism (like socket pair, atomic integers, etc etc), then never mind. — hyde, Jan 30 '13 at 11:54
@AlexChamberlain Uncompression is CPU-intensive operation, so if you target systems which may have several cores, having uncompression happen in parallel, is a very good reason to use threads. — hyde, Jan 30 '13 at 11:56
@hyde Ok, but if you have the same number of threads (or processes) as processors (to avoid oversubscription, for example), you may want to use non-blocking APIs to increase throughput. — Alex Chamberlain, Jan 30 '13 at 12:08

Mark Adler · Accepted Answer · 2013-01-31T15:38:59.133

1

If it's just gzip decompression you need and not tar extraction, then you can use zlib. It is likely already on your system as zlib.h (which is also the documentation) and libz.so.

If you also need tar extraction, then you can try libtar.

edited Jan 31 '13 at 15:38

answered Jan 30 '13 at 15:59

Mark Adler

101,978
13
118
158

The thing is, I do need tar extraction. Is there an easy stream based library for this? – abergmeier Jan 31 '13 at 08:47
I think this is a valid answer, since zlib does at least half of what I try to do. I asked another question [here](http://stackoverflow.com/questions/14622669/extract-tar-in-memory-and-nonblocking) to perhaps also get the tar part done. – abergmeier Jan 31 '13 at 09:46

Extract gzip via passive library

1 Answers1