Zero byte files after XML download in node.js

Question

I have the following code snippet I am currently employing attempting to get Yahoo weather XML files:

// This script requires request libraries.
// npm install request

var fs = require('fs');
var woeid_array = fs.readFileSync('woeid.txt').toString().split("\n");
var grabWeatherFiles = function (array) {
//var http = require('http');
//var fs = require('fs');

array.forEach( 
function(element)  {
    var http = require('http');
    var file_path = 'xml/' + element + '.xml';
    console.log(file_path);
    var file = fs.createWriteStream(file_path);
    var request = http.get('http://weather.yahooapis.com/forecastrss?w=' + element, function(response) {
        response.pipe(file);

    });

});

};
grabWeatherFiles( woeid_array );

This code snippet is successful at downloading the XML files. However, if I attempt to read the files and get the XML data inside a string so I can parse it, the files are 0'd out. Is node.js not writing this correctly? This happened both on my Mac and on c9.io. Any tips would be lovely. I am quite stuck in this part.

score 0 · Answer 1 · answered Jul 18 '13 at 04:03

You are using the wrong function. fs.writeFile takes at least three arguments filename, data and callback. You cannot pipe to it. It simply writes data to the filename and executes callback when done.

What you need is fs.createWriteStream, which takes path (apart from extra options). It creates a writeable stream, to which you can pipe into the response.

score 0 · Answer 2 · edited May 23 '17 at 12:21

These are the steps I used to do the job and it works. Created a folder named xml in the same level where *.js file resides. Created woeids.txt file with some valid woeids from http://woeid.rosselliot.co.nz/lookup/london

created modified version of your code with path definition to use __dirname (useful explanation of it : What is the difference between __dirname and ./ in node.js?) and put code in sample.js:

// This script requires request libraries.
// npm install request

var fs = require('fs');
var woeid_array = fs.readFileSync(__dirname + '/woeids.txt').toString().split("\n");
var grabWeatherFiles = function (array) {
    array.forEach( 
    function(element)  {
        var http = require('http');
        var file_path = __dirname + '/xml/' + element + '.xml';
        console.log(file_path);
        var file = fs.createWriteStream(file_path);
        var request = http.get('http://weather.yahooapis.com/forecastrss?w=' + element, function(response) {
            response.pipe(file);

        });

    });
};
grabWeatherFiles( woeid_array );

Run it through terminal node sample.js and it populated the xml folder with the proper xml files.

Zero byte files after XML download in node.js

2 Answers2