nodejs sends empty response before data retrieved from mongodb

Question

When I use jQuery ajax to retrieve data from nodejs (with express4), nodejs sends empty response back before data loaded from mongodb.

This is the message from nodejs console:

GET /query?uid=1 - - ms - -

And this is the error message from Chrome console:

GET http://192.168.1.105:3000/query?uid=1 net::ERR_EMPTY_RESPONSE

I can confirm that data are correctly loaded from mongodb because data can be printed on nodejs console after nodejs sent the empty response. And this is exactly the problem. Why nodejs sends reponse to client before data have been prepared?

I know nodejs is asynchronous and I pay much attention to this great feature, but I still have this problem.

This is my client code:

$.getJSON('/query', {uid:1}, function(response) { console.log('finished!'); });

And this is my server code:

var express = require('express');
var mongodb = require('mongodb');
var GeoJSON = require('geojson');
var strftime = require('strftime');
var router = express.Router();
var MongoClient = mongodb.MongoClient;

router.get('/query', function(req, res, next) {
    var url = "mongodb://localhost/example_db";
    var collection_name = "example_collection";

    var poi = req.query.poi ? req.query.poi.split("||") : null;
    var time = req.query.time;
    var uid = req.query.uid;

    var condition = {};
    if (poi) condition.poiname = {$in: poi};
    if (time) condition.checkin_time = {$gte:new Date(time.start_time), $lte:new Date(time.end_time)};
    if (uid) condition.uid = parseInt(uid);

    MongoClient.connect(url, function(err, db) {
        if (err) console.log('connection error');

        var collection = db.collection(collection_name);
        collection.find(condition).sort({checkin_time:1}).toArray(function(err, result) {
            if (err) {
                console.log(err);
                return res.send('error!');
            }

            if (!result) return res.send('no data');

            //This line prints the result after empty response has been sent.
            console.log(result);

            var data = {};
            data['geojson'] = GeoJSON.parse(result, {Point:'loc', include:['poiname', 'uid', 'checkin_time']});
            res.json(data);
            db.close();
    });
});

My data are a little bit large, 12G stored in mongodb. So it usually takes about 3 minutes or more to complete the query. When I use findOne to retrieve only a single document, this is no problem.

Does the data size cause the problem?

score 0 · Answer 1 · answered Aug 27 '15 at 08:54

0

Try GeoJSON.parse with callback

        var data = {};
        GeoJSON.parse(result, {Point:'loc', include:['poiname', 'uid', 'checkin_time']}, function (geojson) {
            data['geojson'] = geojson;
            res.json(data);
            db.close();
        });

answered Aug 27 '15 at 08:54

Arūnas Smaliukas

3,231
6
27
46

I modified and the problem still exists. I also notice that empty responses are all sent about 20 seconds after ajax executed. – kikyoakane Aug 27 '15 at 09:11
Maybe it's your servers Timeout? Do you use plain nodejs, or Nginx behind it? – Arūnas Smaliukas Aug 27 '15 at 09:12
Also, when you make client request, you can check in your development console (Firefox firebug for example) response headers. It should be clear if it was timeout or success from status code (`200` - success, `5xx` - error/timeout) – Arūnas Smaliukas Aug 27 '15 at 09:14
I think you get some kind of error with `GeoJSON.parse`.. did you try to `console.log(data['geojson'])` to see if it was successfully parsed? – Arūnas Smaliukas Aug 27 '15 at 09:16
I think the problem lies in server timeout. There is a relevant problem. [http://stackoverflow.com/questions/12651466/how-to-set-the-http-keep-alive-timeout-in-a-nodejs-server](http://stackoverflow.com/questions/12651466/how-to-set-the-http-keep-alive-timeout-in-a-nodejs-server) I have googled how to modify the default timeout value of nodejs, but it seems there is no easy way to achieve that. – kikyoakane Aug 27 '15 at 10:18

nodejs sends empty response before data retrieved from mongodb

1 Answers1