0
const zip = {
            method: 'GET',
            url: distURL + "/zips/" highestNumber + ".zip",
            responseType: 'arraybuffer'
        }
        axios(zip).then(response => {
            console.log("Downloading ZIP: " + highestNumber )
            fs.writeFileSync("./zips/" + highestNumber + ".zip", Buffer.from(response.data));
        })

I have this script above, it downloads a ZIP file from a server and there are ZIP files named as 0.zip, 22.zip, 15.zip and on...

I wonder how can I download the highest available (200) ZIP file and save it?

I have been trying to figure this out for hours but I'm straight stuck.

brian rockson
  • 41
  • 5
  • Get the filenames. Get the filenames without the path and without the ".zip" extension. Parse them into integers and keep the largest one. Convert that to a string variable named "highestNumber". – Andrew Morton Nov 12 '20 at 21:24
  • But I don't know the file names and I just want to download the highest numbered ZIP file that is available. "Some" numbers before the highest one are not available. – brian rockson Nov 12 '20 at 21:25
  • Are they numbered in chronological order, by chance? I.e., highest numbered file is also the latest-dated? If so, you could use something here: https://stackoverflow.com/questions/10559685/using-node-js-how-do-you-get-a-list-of-files-in-chronological-order – Marc Nov 12 '20 at 21:25
  • They just start on 1 till 20. The thing is I don't know the highest number that exists or all of the numbers. – brian rockson Nov 12 '20 at 21:27
  • Guys, I am downloading files from a URL or CDN that I don't own. I don't know the available numbers so there can be more than 20 available numbers but I want to get the highest one. – brian rockson Nov 12 '20 at 21:28
  • 1
    @brianrockson Oh, do you mean that it is a remote server (i.e. not the one your code is running on)? Then you cannot know unless the remote server provides some way of telling your code. – Andrew Morton Nov 12 '20 at 21:28
  • @brianrockson Is there a fixed list of the filenames which *could* exist? If so, start at the latest and keep trying until one is successful. – Andrew Morton Nov 12 '20 at 21:31

2 Answers2

0

Try to read all files and find the highest numbers from that

fs.readdir(directoryPath, function (err, files) {
    //handling error
    if (err) {
        return console.log('Unable to scan directory: ' + err);
    } 
    //listing all files using forEach
    let alltNumbers = new Array();
    files.forEach(function (file,index) {
        // put the file number in array and thenfind thehighest value
        let fileNumber = file.split(".zip");
        alltNumbers[index] = filefileNumber[0];
    });
});
Abhishek
  • 36
  • 4
0

if the server supports file listing

you can list files and as previous answers pointed out,
figure the maximal number.

If it don't

Most servers won't expose directory listing as it can cause a security issues. But because you know the pattern of file you seek, you can use a "trial & error" method by try requesting a file and check if it exist RetCode 200, or not there RetCode 404.
You might want to terminate the connection upon STATUS reception and not complete the entire stream as it can be sllllooooowwwwww

use it somewhat like a binary search (in order of 100)... and narrow till you find the one that has next.

e.g.

// range seek phase
req 100 => there
req 1000 => not there
// Range = [100, 1000)

// binary search phase
req 550 => there  // 550 is mid point of (100, 1000)
// Range = [550, 1000)
req 775 => not there
// Range = [550, 775)
req 687 => there
...

Note: this isn't a great method to design a system, you rather expose a /log/last route to download the last log file or a /log/list to list the available logs. (maybe provide FTP Access for the log folder?)

But from a "hacking" perspective, it's a great method which is used seldomly.

Tomer W
  • 3,395
  • 2
  • 29
  • 44