Node.js: http request timing out after 1 minute

Question

I am using Node.js (version 0.10.28) to make an http.request to my Ruby API for a large amount of data (21,000 rows from a PostgreSQL database). The problem is that the request seems to time out and return a 404 error after 1 minute. I know this is true as the Ruby API returns 61 seconds for the request, and I time how long the Node's request takes (results in 60 seconds and the 404). However, if I wget or use jQuery's $.ajax with a timeout of 4 minutes, I can get my 21,000 rows. So clearly it can't be a 404, as other ways of getting the data work.

I'm a little confused too because everywhere I look, the http.request isn't supposed to time out until after 2 minutes, according to:

Node.js docs

GitHub issue #1

GitHub issue #2

I have tried several things to get this working, including: setting express's middleware; listening for the socket timeout and resuming the request; and setting the timeout to 0, so there's no timeout. Unfortunately, none of these ways worked, or at least from what I've understood.

For some clarity, here's my code... setting that limit means return only 18,000 records, and that seems to be the cut off point for the API taking longer than 60 seconds:

var http = require('http');
var options = {
  path: '/api/records.json?limit=18000',
  method: 'GET',
  host: 'localhost',
  requestCert: true,
  rejectUnauthorized: false
};
var req = http.request(options, function(res) {
  var endDate = new Date();
  console.log('done', endDate - startDate);
  var output = [];
  res.on('data', function(chunk) {
    output.push(chunk);
  });
  res.on('end', function() {
    var data = output.join('');
    console.log(data);
    return {data: data, success: true};
  });
});
req.on('socket', function(socket) {
  socket.setTimeout(0);  // no timeout
  socket.on('timeout', function() {
    socket.resume();  // tried resuming the timeout
  });
});
req.end();
var startDate = new Date();
console.log('starting', startDate);

The API URL is valid and works, as I've verified that through wget and $.ajax, so how do I fix the timeout issue in Node?

UPDATE

Upon further inspection, if I take out socket.setTimeout(0); and try to get into the socket.on('timeout', I don't actually get into the callback... which is odd, but it's always 60 seconds before I get the 404

Have you tried inspecting your traffic with Wireshark to make sure that Node isn't actually getting a response? — Stuart P. Bentley, Jul 11 '14 at 14:45
Also, it might be important to note that the documentation you linked to describes *server* timeouts for Node. The documentation pertaining to *request* timeouts is http://nodejs.org/api/http.html#http_request_settimeout_timeout_callback — Stuart P. Bentley, Jul 11 '14 at 15:10
Yeah, sorry, I've tried so many things that I just got lost in the documentation... I have tried `req.setTimeout(0);` but no dice. I also thought I had read that `http` basically is `net`, which implements `server`. — incutonez, Jul 11 '14 at 15:17
Upon further inspection, if I take out `socket.setTimeout(0);` and try to get into the `socket.on('timeout'`, I don't... which is odd, but it's always 60 seconds before I get the 404. — incutonez, Jul 11 '14 at 15:36
A socket timeout wouldn't trigger a 404 response anyway, which is why I suggest you watch the actual request on the wire. — Stuart P. Bentley, Jul 11 '14 at 15:51
Right, I think the 404's the key, which I believe is an nginx setting that I need to change... trying it now. — incutonez, Jul 11 '14 at 15:55

score 5 · Accepted Answer · edited Nov 13 '17 at 11:31

5

It's an nginx issue, as I'm using it to proxy through to my API, so it was a simple change of the proxy's configuration by setting proxy_read_timeout 180s.

edited Nov 13 '17 at 11:31

Kalamarico

5,466
22
53
70

answered Jul 11 '14 at 15:59

incutonez

3,241
9
43
92

2

You are a life saver. Indeed the default for [proxy_read_timeout](http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) is 60 seconds. After that nginx will abruptly close the request regardless of whether NodeJS is done processing or not. – mae Jun 15 '21 at 08:05

score 2 · Answer 2 · answered Sep 26 '20 at 13:34

2

Friends have had the same problem After working for 15 days, the source of the problem is "server.headersTimeout = 7200000;" you can fix it by adding this code

answered Sep 26 '20 at 13:34

İsmail Gron Yağcı

29
1

Kalamarico · Answer 3 · 2018-02-09T16:38:57.543

As you said it's a nginx problem, but probably if you don't set in the nginx config the proxy parameters nginx default depends on the operative system, because in Ubuntu 16.04 I haven't any problem with a 10min timeout and without any proxy parameter, but in centos and redhat I saw the 1 min timeout restriction.

I add a configuration example because there are two parameters more than proxy_read_timeout that you need to set:

server {
    listen       80;
    server_name  myserver;

    location / {
        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

        proxy_connect_timeout      180s;
        proxy_send_timeout         180s;
        proxy_read_timeout         180s;

        proxy_buffer_size          4k;
        proxy_buffers              4 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;
    }
}

For this issue the important parameters are:

proxy_connect_timeout      180s;
proxy_send_timeout         180s;
proxy_read_timeout         180s;

Node.js: http request timing out after 1 minute

3 Answers3