1

I have a zip folder downloaded from s3 bucket. Now I have a json file in my code, and I want to add my JSON file to an existing zip file using node js code.

Is there any pre existing module for doing this in node js?

I tried easy-zip, but I was not able to add the file to an existing zip-folder.

Any idea on this?

Italo Borssatto
  • 15,044
  • 7
  • 62
  • 88
Vishnu Ranganathan
  • 1,277
  • 21
  • 45

4 Answers4

3

I cannot find any any library that will do this inline and modify an already finalized zip. Here's a brute force approach that goes through the following steps:

  1. Create Temp Directory
  2. Extract Zip to Temp Dir
  3. Delete Original Zip
  4. Build Archive and seed with Temp Dir
  5. Callback to append any additional files
  6. Finalize Archive
  7. Delete Temp Directory

It uses extract-zip to unpack the zip and archiver to bundle it back up.

You can see the full source code at this repo KyleMit/append-zip

appendZip.js

// require modules
const fs = require('fs');
const fsp = fs.promises
const archiver = require('archiver');
const extract = require('extract-zip')

async function appendZip(source, callback) {
    try {
        let tempDir = source + "-temp"

        // create temp dir (folder must exist)
        await fsp.mkdir(tempDir, { recursive: true })

        // extract to folder
        await extract(source, { dir: tempDir })

        // delete original zip
        await fsp.unlink(source)

        // recreate zip file to stream archive data to
        const output = fs.createWriteStream(source);
        const archive = archiver('zip', { zlib: { level: 9 } });

        // pipe archive data to the file
        archive.pipe(output);

        // append files from temp directory at the root of archive
        archive.directory(tempDir, false);

        // callback to add extra files
        callback.call(this, archive)

        // finalize the archive
        await archive.finalize();

        // delete temp folder
        fs.rmdirSync(tempDir, { recursive: true })

    } catch (err) {
        // handle any errors
        console.log(err)
    }
}

Usage

async function main() {
    let source = __dirname + "/functions/func.zip"

    await appendZip(source, (archive) => {
        archive.file('data.json');
    });
}

In the callback, you can append files to the archive using any of the methods available with node-archiver, for example:

// append a file from stream
const file1 = __dirname + '/file1.txt';
archive.append(fs.createReadStream(file1), { name: 'file1.txt' });

// append a file from string
archive.append('string cheese!', { name: 'file2.txt' });

// append a file from buffer
const buffer3 = Buffer.from('buff it!');
archive.append(buffer3, { name: 'file3.txt' });

// append a file
archive.file('file1.txt', { name: 'file4.txt' });

// append files from a sub-directory and naming it `new-subdir` within the archive
archive.directory('subdir/', 'new-subdir');

// append files from a sub-directory, putting its contents at the root of archive
archive.directory('subdir/', false);

// append files from a glob pattern
archive.glob('subdir/*.txt');

Further Reading

KyleMit
  • 30,350
  • 66
  • 462
  • 664
1

Adm-zip provides the facility to add a file to a zipped folder directly. Browsing through their api, they have two ways to add a file to a zipped folder. Firstly by adding a file from the file system and second by providing the filename and content (Buffer).

addLocalFile(localPath, zipPath) // Adds a file from the disk to the archive
addFile(entryName, content, comment, attr) // Allows you to programmatically create a entry (file or directory) in the zip file.

I tried it on my local machine and it's working without unzipping the existing directory. But the only problem is that file added exists in memory, we need to flush it back to file system if we need to see the changes permanently.

// test.zip (contents)
//   - pdfReader.py
//   - white-cuts-on-black-forever.py

var AdmZip = require('adm-zip');
var zip = new AdmZip("test.zip"); // Could also give a buffer here (if it's in memory)

var data = Buffer.from(JSON.stringify({"a": 3, "b": "hello"}));
zip.addFile("data.json", data);   // New file created from contents in memory

zip.addLocalFile("trigger.json"); // Local file in file system

zip.writeZip("test.zip");

// test.zip (contents after program execution)
//   - data.json
//   - trigger.json
//   - pdfReader.py
//   - white-cuts-on-black-forever.py
TheChetan
  • 4,440
  • 3
  • 32
  • 41
0

Here's an example from Append Files to Existing Zip w/out Rewriting Entire Zip

Since the above is a bit longer way, the only other option would be to use node-stream-zip, create a new object after reading the existing one, add your files and then re-create the zip. You can choose to delete the original zip then. However, it would not be 'appending' in-effect. It is a trade-off which you can comfortably make though since streaming is involved.

KyleMit
  • 30,350
  • 66
  • 462
  • 664
Anshul Verma
  • 1,065
  • 1
  • 9
  • 26
0

I would recommend JSZip for this task. As i understand you want to download zip file from S3 bucket merge your JSON document in the existing structure of the zip (this means actually open and read it).

Using JSZip this can be done in 4 steps