node.js - use archiver where output is buffer

Question

I want to zip a few readeableStreams into a writableStream. the purpose is to do all in memory and not to create an actual zip file on disk.

for that i'm using archiver

        let bufferOutput = Buffer.alloc(5000);
        let archive = archiver('zip', {
            zlib: { level: 9 } // Sets the compression level.
        });
        archive.pipe(bufferOutput);
        archive.append(someReadableStread, { name: test.txt});
        archive.finalize();

I get an error on line archive.pipe(bufferOutput);.

This is the error: "dest.on is not a function"

what am i doing wrong? Thx

UPDATE:

I'm running the following code for testing and the ZIP file is not created properly. what am I missing?

const   fs = require('fs'),
    archiver = require('archiver'),
    streamBuffers = require('stream-buffers');

let outputStreamBuffer = new streamBuffers.WritableStreamBuffer({
    initialSize: (1000 * 1024),   // start at 1000 kilobytes.
    incrementAmount: (1000 * 1024) // grow by 1000 kilobytes each time buffer overflows.
});

let archive = archiver('zip', {
    zlib: { level: 9 } // Sets the compression level.
});
archive.pipe(outputStreamBuffer);

archive.append("this is a test", { name: "test.txt"});
archive.finalize();

outputStreamBuffer.end();

fs.writeFile('output.zip', outputStreamBuffer.getContents(), function() { console.log('done!'); });

score 4 · Answer 1 · answered Apr 11 '18 at 04:04

In your updated example, I think you are trying to get the contents before it has been written.

Hook into the finish event and get the contents then.

outputStreamBuffer.on('finish', () => {
  // Do something with the contents here
  outputStreamBuffer.getContents()
})

score 2 · Answer 2 · answered Nov 18 '19 at 17:50

2

By adding the event listener on archiver works for me:

archive.on('finish', function() {
   outputStreamBuffer.end();
   // write your file
});

answered Nov 18 '19 at 17:50

Z Li

21
1

score 1 · Answer 3 · answered Jul 19 '17 at 10:28

1

A Buffer is not a stream, you need something like https://www.npmjs.com/package/stream-buffers

answered Jul 19 '17 at 10:28

Dave Irvine

384
4
14

Thx Dave, i've added a followup question if you can help with as well – Yoni Mayer Jul 19 '17 at 11:43

score 1 · Answer 4 · answered Jul 19 '17 at 11:52

1

As for why you are seeing garbage, this is because what you are seeing is the zipped data, which will seem like garbage.

To verify if the zipping has worked, you probably want to unzip it again and check if the output matches the input.

answered Jul 19 '17 at 11:52

Dave Irvine

384
4
14

i just noticed that it actually get stuck on "outputStreamBuffer.getContents().pipe(output);" and doesn't procced – Yoni Mayer Jul 19 '17 at 12:33
1

Ah, because, again, `outputStreamBuffer.getContents()` gives you a Buffer not a Stream, so it has no `pipe()` function. Just do `fs.writeFile('output.zip', outputStreamBuffer.getContents(), function() { console.log('done!'); });` – Dave Irvine Jul 19 '17 at 12:37
umm... still not working for me. i've updated my question if you can have a look. thx a lot for your help – Yoni Mayer Jul 20 '17 at 13:44

node.js - use archiver where output is buffer

4 Answers4