Download zip with axios and unzip with adm-zip in memory (electron app)

Question

I need to download a file with axios and unzip it in memory in an electron app.

I read in some SO threads (e.g.), that adm-zip supports byte buffer constructor, but I can not see this in the docs. When I extract the content, it behaves like the array is empty, but it is not. It just does create a file and does not throw any errors I do not want to use request, as the api is marked deprecated. My code is this:

const axios = require("axios");
const AdmZip = require('adm-zip');
   
const url = "http://update-service.test.w3champions.com/api/maps";
const body = await axios.get(url, {
    responseType: 'arraybuffer'
});
const data = body.data;
const zip = new AdmZip(data);
zip.extractAllTo(to, true);

I feel super stupid, because I had it one time working and then changed something and now I do not seem to find the error again :/ I sadly did not commit the working state...

edit: So, we figured it out: Electron does some weird stuff that returns an Array Buffer instead of a Buffer, that adm-zip would need. As I am lazy added the package arraybuffer-to-buffer and now the code works:

const arrayBufferToBuffer = window.require('arraybuffer-to-buffer');
const url = `${this.updateUrl}api/${fileName}?ptr=${this.isTest}`;
const body = await axios.get(url, {
    responseType: 'arraybuffer'
});

const buffer = arrayBufferToBuffer(body.data);
const zip = new AdmZip(buffer);
zip.extractAllTo(to, true);

For anyone reading this question looking for the answer, please look at the edit here. `arrayBufferToBuffer` is the same as `const buffer = Buffer.from(body.data);` — Wyetro, Feb 18 '22 at 02:08

Matus Dubrava · Answer 1 · 2020-08-06T20:41:08.630

9

It works the same with axios. The code below is a working example.

const axios = require('axios');
const AdmZip = require('adm-zip');

const f = async () => {
    const url = 'http://update-service.test.w3champions.com/api/webui';
    const body = await axios.get(url, {
        responseType: 'arraybuffer',
    });

    var zip = new AdmZip(body.data);
    var zipEntries = zip.getEntries();

    // search for "index.html" which should be there
    for (var i = 0; i < zipEntries.length; i++) {
        console.log(zip.readAsText(zipEntries[i]));
    }

    // and to extract it into current working directory
    zip.extractAllTo('.', true);
};

f();

edited Aug 06 '20 at 20:41

answered Aug 06 '20 at 20:28

Matus Dubrava

13,637
2
38
54

The zips contain some folders with files, I want to unzip. I updated the URL for another file to make it more clear. – modmoto Aug 06 '20 at 20:30
The code above does just that. `zip.readAsText(zipEntries[i])` will return unzipped files as text. It works for your new url as well except that it contains binary data so it is not very helpful. – Matus Dubrava Aug 06 '20 at 20:34
And i just copied your code and I do not see anything in the log. Also `extractAll` from `adm-zip` does not do anything. Dont know why this is so hard o.O – modmoto Aug 06 '20 at 20:34
It does, you just need to wait because the file is large. So it takes time before it is downloaded. – Matus Dubrava Aug 06 '20 at 20:35
Maybe i should add that this is an electron app, but this should not matter all to much. It also does not have anything to do with file rights, i can create text files from my app on the same location. – modmoto Aug 06 '20 at 20:36
On what location? This is running in memory, not on a disk. – Matus Dubrava Aug 06 '20 at 20:37
I am debugging it, when the get is done, I have the whole array in memory, but it does not get to console or the folder :/ also the webui zip is only a few kb and this also does not extract. – modmoto Aug 06 '20 at 20:37
yes, but the `extractAll` goes to the disk. And as i said, the console log returns an empty string for me. – modmoto Aug 06 '20 at 20:37
Even if it returns and empty response in console, you wouldn't see anything useful there because it is not text. So what exactly are you trying to achieve here? – Matus Dubrava Aug 06 '20 at 20:40
Try to copy paste the code that I have written now and wait till it finishes. – Matus Dubrava Aug 06 '20 at 20:43
I am trying to tell you, that the index.html gets printed out as an empty string in the code you provided. And I dont know why :< I added my repo if you want to reproduce it – modmoto Aug 06 '20 at 20:44
i copied it, the whole thing, the console stays empty – modmoto Aug 06 '20 at 20:45
Here is screenshot if you do not believe me https://www.dropbox.com/s/pslkvrj5yo4zhpo/Capture.JPG?dl=0 I really do not get it, it looks like an exception is just swallowed or the array is empty. – modmoto Aug 06 '20 at 20:53
I see. It has to be electron thing. Running your github code now, will update you if I figure something out. – Matus Dubrava Aug 06 '20 at 21:12
I got it, the other answer pushed me in the right direction. I debugged into the adm files and saw, that it wanted a buffer, but axios delivers an array buffer. Must be an electron speciality xD I fixed it and will post the answer. Thanks for the help though! – modmoto Aug 06 '20 at 21:18

score 3 · Accepted Answer · answered Aug 06 '20 at 20:27

3

check the typeof data, maybe its not buffer.

Adm implementation: https://github.com/cthackers/adm-zip/blob/master/adm-zip.js

answered Aug 06 '20 at 20:27

Harsh Srivastava

376
2
10

It is an array buffer, at least chrome tools say that. – modmoto Aug 06 '20 at 20:54
https://www.dropbox.com/s/fwnufon8vilz1o3/Untitled.png?dl=0 – modmoto Aug 06 '20 at 21:02
Hey, you are right, i just debugged into it and it takes the last else, which is an empty file. Somehow arraybuffer and buffer are not the same. Gonna investigate this. – modmoto Aug 06 '20 at 21:09
If you want to copy my result? I think the answer is better to find, if it is together with the answer. – modmoto Aug 06 '20 at 21:21

Download zip with axios and unzip with adm-zip in memory (electron app)

2 Answers2

Linked