Can not open zip file after downloading through nodejs application because of Invalid CEN error

Question

I need to download and unzip zip archive in my nodejs application. I have this code:

utils.apiRequest(teamcityOptions)
        .then(function (loadedData) {
          var tempDir = tmp.dirSync();
          var tmpZipFileName = tempDir.name + "\\" + 'bob.zip';

          fs.appendFileSync(tmpZipFileName, loadedData);

          var zip;
          try {
            zip = new AdmZip(tmpZipFileName);
          } catch (e) {
            log('Can not create zip, bad data', e);
          }
        });

This code gives me error:

Can not create zip, bad data Invalid CEN header (bad signature)

I am using Windows 7. I can't even open this ZIP file with 7-zip or WinRAR (simple error like corrupted data).

Also, utils.apiRequest function body is:

utils.apiRequest: function (options) {
  var deferred = defer();
  https.get(options, function (request) {
    var loadedData = '';
    request.on('data', function (dataBlock) {
      loadedData += dataBlock.toString('utf8');
    });
    request.on('end', function () {
      deferred.resolve(loadedData);
    })
  });
  return deferred.promise;
}

How can I solve my problem?

PS: I don't have problems using curl :)

It is string. I got it in the apiRequest function by https node module. I will add the code of the function in 1 hour. — Sharikov Vladislav, Jul 20 '16 at 06:27
I added code of `utils.apiRequest` function, so you can see type of `loadedData`. — Sharikov Vladislav, Jul 20 '16 at 06:59

Vincent Schöttke · Answer 1 · 2016-07-20T08:13:02.670

3

The problem is that you're decoding the received data into an utf8 string:

request.on('data', function (dataBlock) {
  loadedData += dataBlock.toString('utf8'); // this is wrong
});

Since a zip file is binary you should use a Buffer.

Here is an example replacement for your utils.apiRequest with Buffer:

utils.apiRequest: function (options) {
    var deferred = defer();
    https.get(options, function (request) {
        var data = []; 
        request.on('data', function (dataBlock) {
            data.push(dataBlock); 
        });
        request.on('end', function () {
            deferred.resolve(Buffer.concat(data));
        });
    });
    return deferred.promise;
}

edited Jul 20 '16 at 08:13

answered Jul 20 '16 at 07:14

Vincent Schöttke

4,416
2
12
12

How exacly I have to write my code? `.toString` is redunant? I tried this. This does not work. – Sharikov Vladislav Jul 20 '16 at 07:16
dataBlock is a Buffer. So you can not use `+=` to append. node does not have an append function for the Buffer so you have to write your own or use the node-bufferjs module. You can also write the loaded data directly with your appendFileSync function or pipe it to the destination file. – Vincent Schöttke Jul 20 '16 at 07:21
Ah. This is answer for my this question http://stackoverflow.com/questions/38466944/can-not-open-zip-file-after-downloading-through-nodejs-application-because-of-in/38474661#comment64351596_38469236. `loadedData` is buffer only when I make request to download zip file? Is this simple string when I do simple API request? Should I change my code in other simple (not `.zip`, just JSON requests) cases? – Sharikov Vladislav Jul 20 '16 at 07:31
The `dataBlock` in the `.on` callback function is always a Buffer. If you need binary data you should not encode it as a string. – Vincent Schöttke Jul 20 '16 at 07:41
I tried this. Code was just `loadedData += dataBlock` and it didn't work. – Sharikov Vladislav Jul 20 '16 at 07:54
Please see my second comment why your += does not work. If you don't want to code all the stuff by hand I highly recommend using the request module inside your `apiRequest` method with the `{encoding: null}` for binary data option. – Vincent Schöttke Jul 20 '16 at 08:03

score 2 · Answer 2 · answered Jul 19 '16 at 21:59

(Adding as an answer so I can post the code snippet)

@vincent is on the right track I think - sounds like you're not writing the data as binary to the file. It's often easier to just pipe a download request straight to a file:

var http = require('http');
var fs = require('fs');
var AdmZip = require('adm-zip')

var tmpZipStream = fs.createWriteStream('bob.zip');
var request = http.get('http://example.com/example.zip', function(response) {
  response.pipe(tmpZipStream);
});

tmpZipStream.on('close', function() {
  var zip;
  try {
    zip = new AdmZip('bob.zip');
  } catch (e) {
    console.log('Can not create zip, bad data', e);
  }
})

Without knowing where utils.apiRequest comes from it's hard to say if this is workable for you, but hopefully it helps.

One question: do I have to close stream manually somehow? After using your approach my application doesn't stop. Looks like waiting for some infinity timeout — Sharikov Vladislav, Jul 20 '16 at 18:51

Can not open zip file after downloading through nodejs application because of Invalid CEN error

2 Answers2

Linked