How can I stream a zip in a zip stream download?

Question

I want to be able to archive (zip, no compression needed, but that is a plus),
in memory and stream it, the problem is that I want to create an inner zip in
the zip I am streaming, like so:

Files:

a.txt, b.txt, c.txt

Stream download should look like:

my.zip {
  a.txt
  inner.zip {
   b.txt, c.txt
  }
}

Notes, I have to stream the files because I have no HD storage available and I can't have all the files in memory either (that is why I am streaming them)

---

Here Is a normal zip stream I managed to get working (without the inner zip streamed yet):

<?php
require 'ZipStream.php';

$zip = new ZipStream\ZipStream('my.zip');
$zip->addFileFromPath('a.txt', 'a.txt');
// I want to add inner.zip here and stream it too only from memory
$zip->finish();

Answer 1

You can run a small example to see if using ZipArchive could help you on this. Create 3 empty .txt files named a,b,c in the root directory of the example.

PHP

function RandomString($file)
{
    $characters = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ';
    $randstring = file_get_contents($file);
    for ($i = 0; $i < 2000000; $i++) {
        $randstring .= $characters[rand(0, strlen($characters))];
    }
    file_put_contents($file,$randstring);
    return true;
}
// fill the 3 files with data up to 2Mb per run
RandomString("a.txt");
RandomString("b.txt");
RandomString("c.txt");

 $zip = new ZipArchive;
    $res = $zip->open('inner.zip', ZipArchive::CREATE);
    if ($res === TRUE) {
    $zip->addFile('b.txt', 'b.txt');
    $zip->addFile('c.txt', 'c.txt');
    $zip->close();
        $resMy = $zip->open('my.zip', ZipArchive::CREATE);

        if ($resMy === TRUE) {
            $zip->addFile('a.txt', 'a.txt');
            $zip->addFile('inner.zip', $contents);
            $zip->close(); 
            unlink('inner.zip');            
            $file_name = basename("my.zip");
            header("Content-Type: application/zip");
            header("Content-Disposition: attachment; filename=$file_name");
            header("Content-Length: " . filesize("my.zip"));
            readfile("my.zip");
            unlink("my.zip");
            exit;
        } else {
            echo 'failed to create my.zip';
        }
    } else {
        echo 'failed to create inner.zip';
    }

The above example was tested and for 55Mb of data in 3 files the output produced was a zip file of ~300Kb.

Answer 2

Well, this kind of works. I wanted to post it although I decided to fix the problem in another way. But just for future reference this may be helpful for someone.

<?php
$zip_header_outer = "\x1f\x8b\x08\x08i\xbb\xb9X\x00\xffinner.zip\x00";
$zip_header_inner = "\x1F\x8B\x08\x08\x69\xBB\xB9\x58\x00\xFF";
$compression_level = 0;
$download_headers = true;
$download_file_name = 'my.zip';
$inner_file_name = 'inner.zip';
$first_level_file = 'a.txt';
$second_level_file = 'b.txt';


$fout = fopen("php://output", "wb");
if ($fout === FALSE) {
    die('Problem outputting');
}

if ($download_headers) {
    header("Content-type: application/octet-stream");
    header("Content-Disposition: attachment; filename=\"" . $download_file_name . "\"");
}

function add_to_inner_zip($filename, $path, &$fout, &$fsize_outer, &$hctx_outer) {
    $zip_header_inner = "\x1F\x8B\x08\x08\x69\xBB\xB9\x58\x00\xFF";

        fwrite($fout, $zip_header_inner);
        hash_update($hctx_outer, $zip_header_inner);
        $fsize_outer += strlen($zip_header_inner);
        $hctx_inner = hash_init("crc32b");
        $fsize_inner = 0;

        // Inner Add file name
        $file_name = str_replace("\0", "", basename($filename));
        $data = $file_name."\0";
        fwrite($fout, $data, 1+strlen($data));
        hash_update($hctx_outer, $data);
        $fsize_outer += strlen($data);

        // Start inner.zip contents
            // Inner Add file data * STREAM CHUNKS HERE * 
            $file_data = file_get_contents($path);
            $clen = strlen($file_data);
            hash_update($hctx_inner, $file_data);
            $fsize_inner += $clen;
            /*hash_update($hctx_member, $file_data);
            $fsize_member += $clen;*/

            // Actually encode the chunk and add to the main stream
            $gziped_chunk = zlib_encode($file_data, ZLIB_ENCODING_RAW);
            fwrite($fout, $gziped_chunk);
            hash_update($hctx_outer, $gziped_chunk);
            $fsize_outer += strlen($gziped_chunk);

        // Close the inner zip 
    $crc = hash_final($hctx_inner, TRUE);
    $zip_footer = $crc[3].$crc[2].$crc[1].$crc[0] . pack("V", $fsize_inner);
    fwrite($fout, $zip_footer);

    // update outer crc + size
    hash_update($hctx_outer, $zip_footer);
    $fsize_outer += strlen($zip_footer);
}


// Outer
fwrite($fout, $zip_header_outer);
$fltr_outer = stream_filter_append($fout, "zlib.deflate", STREAM_FILTER_WRITE, $compression_level);
$hctx_outer = hash_init("crc32b");
$fsize_outer = 0;

    add_to_inner_zip($first_level_file, $first_level_file, $fout, $fsize_outer, $hctx_outer);
    
stream_filter_remove($fltr_outer);
$crc = hash_final($hctx_outer, TRUE);
fwrite($fout, $crc[3].$crc[2].$crc[1].$crc[0], 4);
fwrite($fout, pack("V", $fsize_outer), 4);

Plus, I know the code sucks, and is hacky - but the situation asked for it :P