2

I have created a function in JavaScript that downloads an image from a URL and writes the image to disk. The function works perfectly for most of the images I download, however there are 30 or so (out of 400) that are corrupted and will not open. The files are created but there is no extension associated and the details show that the file is 0 kB. I know that the files from the source are not corrupted because I have a working solution for this in C# which works for all files.

var download = function(url, destination, callback) {
    fs.exists(destination, function(exists){
        if (!exists){
            http.get(url, function(response) {
                var imageData = '';
                response.setEncoding('binary');
                response.on('data', function(chunk){
                    imageData += chunk;
                });
                response.on('end', function(){
                   fs.writeFile(destination, imageData, 'binary', function(error){
                      if(error){
                          console.log(error);
                      } else{
                          callback();
                      } 
                   });
                });
            });
        }
    });
};

I am working on a Windows machine, not sure if that could some how be an issue, so I figure I’d mention it.

dakab
  • 5,379
  • 9
  • 43
  • 67
ntraylor
  • 53
  • 6
  • 2
    FWIW you can simply stream the file with: `response.pipe(fs.createWriteStream(destination)).on('finish', callback);` instead of buffering the entire file and using `fs.writeFile()`. – mscdex Mar 06 '16 at 18:18
  • Try [this answer](http://stackoverflow.com/a/21024737/2365824) for an alternative method using a Buffer array. – Perspectivus Mar 06 '16 at 19:04
  • Possible duplicate of [Getting binary content in node.js with http.request](http://stackoverflow.com/questions/17836438/getting-binary-content-in-node-js-with-http-request) – bolav Mar 06 '16 at 19:22

1 Answers1

0

So it turned out that I wasn't checking for illegal characters in the path string. In my C# solution I was using a different file name. Once I removed the illegal characters with regex, all is well.

ntraylor
  • 53
  • 6
  • What were the illegal characters? How did you remove them? – dakab May 08 '16 at 12:45
  • \ / : * ? " < > | These characters are invalid and cannot be included on a Windows OS. So I used RegExp() to remove them. There are other ways to do this, but this solution worked best for me. – ntraylor May 19 '16 at 15:33