25

I am trying to use pigz for parallel compress and decompress. I have found a way to compress in parallel using the following command:

tar cf - /input/dir | pigz > output_file.tar.gz

What is the equivalent command to decompress? I need to be able to specify the filename and the output directory path in the command for it to be useful for me.

bryant1410
  • 5,540
  • 4
  • 39
  • 40
user788171
  • 16,753
  • 40
  • 98
  • 125
  • http://zlib.net/pigz/pigz.pdf – cHao Feb 20 '13 at 12:44
  • 7
    You can't decompress in parallel, though. – cHao Feb 20 '13 at 12:47
  • 4
    _pigz_ stands for _p_ arallel _i_ mplementation of _gz_ ip, and is an anagram of gzip. It is pronounced pig-zee. – Mark Adler Feb 20 '13 at 16:22
  • This question does not belong in stackoverflow. – Mark Adler Feb 20 '13 at 16:26
  • pigz -d < test.tar.gz | tar xf - -C /this/directory – user788171 Feb 20 '13 at 17:39
  • 5
    @MarkAdler: It's going to be pronounced "pigs", regardless of what a couple of geeks want to throw up on a web page. They are not above the rules of phonics. (They could have been, if they'd capitalized or something. But if they want to make it look like a word, it's gonna get pronounced like one.) And really? Anagrams are a thing, that people actually do? I hadn't seen a live person do it before; hell, i'm still about 80% convinced it's just some BS they made up for detective shows. :) – cHao Feb 21 '13 at 09:41
  • @cHao - I'm pretty sure when the author tells you how something's pronounced, you take his word for it. – atxdba Jun 30 '14 at 18:40
  • 3
    @atxdba: Tell that to the guys who invented GIF. They have told people how it's pronounced, and everyone i've talked to openly disregards it. – cHao Jun 30 '14 at 18:59
  • @cHao why should you not be possible to decompress in parallel? Comparing `gunzip -c disk_image.gz | ...` and `pigz -dc disk_image.gz | ...`, `pigz` runs about 3x faster. – Cadoiz Jan 19 '21 at 23:44
  • 2
    @Cadoiz: Because pigz didn't have an option for it at the time of writing. As for why it wasn't an option at the time, gzip relies on previous contents to build the table to (de)compress future contents. You need to process what came before to understand what comes after. Which makes the process of decompression inherently serial. That changes if you can find boundaries between chunks, but that's nowhere near as simple for decompression as for compression. – cHao Jan 20 '21 at 11:45
  • 1
    @Cadoiz: Now, there are parts that can be done while the decompression is going on (reading the next bit of data, writing out results, etc), and doing those on another thread can speed things up if the decompression takes longer than the I/O. If it doesn't, though, you're I/O bound, and past a certain point, parallelization can actually slow you down a bit. – cHao Jan 20 '21 at 11:52

4 Answers4

41

Use pigz -dc for decompression to stdout. Then use > as you are in your example to direct the output to the desired path and file.

You can type just pigz for command options help.

Mark Adler
  • 101,978
  • 13
  • 118
  • 158
20

You're probably looking for something along the lines of:

pigz -dc archive.tar.gz | tar xf -

but noting Mark Adler's (legendary, at this point) original post, pigz does not utilize multiple cores for decompression. However, it does utilize additional cores for reading, writing, and some additional calculations, which do yield a moderate performance increase over gzip. Enjoy!

anothernode
  • 5,100
  • 13
  • 43
  • 62
itsmisterbrown
  • 471
  • 4
  • 3
10

tar -I pigz -xvf compressed_file.tar.gz

synkro
  • 414
  • 7
  • 9
8

For some reason, pigz doesn't autocomplete ".gz" compressed files, but if you type the names of your files pigz finds them.

To decompress and keep your compressed files use: pigz -dk yourfilename.gz. If you don't need the compressed versions use just pigz -d yourfilename.gz.

pigz --help shows all the options available.

Daniel Kumazawa Morais
  • 81
  • 1
  • 1
  • So the presence of `-k` determines if the archive `yourfilename.gz` is kept or not? Or what does it do? – Cadoiz Jan 19 '21 at 23:34
  • man pigz|grep -A1 -- '-k' `-k --keep` `Do not delete original file after processing.` – thoth Jan 03 '22 at 16:00