0

I have a node express/ socket application where my express server makes several API calls with node-rest-client looping through elements in var jobs and when each finishes, it sends the data via socket io to the client. However, every now and then i get a socket hang error after about 1000 or so API calls.

events.js:182
      throw er; // Unhandled 'error' event
      ^

Error: socket hang up
    at createHangUpError (_http_client.js:345:15)
    at Socket.socketOnEnd (_http_client.js:437:23)
    at emitNone (events.js:110:20)
    at Socket.emit (events.js:207:7)
    at endReadableNT (_stream_readable.js:1059:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)

How do you error handle these errors? Or perhaps my initial attempt to code this function is poor, and in which case, any suggestion on how to make multiple API calls and emit the results to all sockets connect? (Requirement, the only way i can get the information is through making these API calls).

Server:

setInterval(function(){
  var jobs = ['J1', 'J2', 'J3', 'J4'];
  var full_data = {};
  for(var i = 0; i < jobs.length; i++){
    client.get("MY URL", function (data, response) {
        io.sockets.emit('progressbar', data);
      });
  }
  console.log(full_data);

}, 5000)

Where, 'progressbar' is the client function listening for data.

BLang
  • 930
  • 3
  • 16
  • 35
  • Is the socket error happening on the socket.io connections or on the `client.get()` request? – jfriend00 Feb 20 '18 at 03:34
  • I narrowed it down to the client.get() request, i wasn't sure at first since it was stated "socket hang up" error, and i just assumed that meant socket.io, but i posted my working solution below. – BLang Feb 21 '18 at 18:36
  • Are you really only doing four requests like this every 5 seconds? Or is the real `jobs` array a lot longer? – jfriend00 Feb 21 '18 at 19:41
  • The real jobs array can grow much larger. – BLang Feb 21 '18 at 20:00
  • I ask because with a larger array, you may just have too many requests in flight at the same time, either too many for your end of the connection or more than the host will allow on the other end. If so, you may want to use something like is shown here: https://stackoverflow.com/questions/33378923/make-several-requests-to-an-api-that-can-only-handle-20-request-a-minute/33379149#33379149. – jfriend00 Feb 21 '18 at 20:16
  • You also need to make sure that it doesn't take more than 5 seconds to process all the requests or you'll be starting the next iteration before finishing the prior one which makes the request overload problem worse. – jfriend00 Feb 21 '18 at 20:16

2 Answers2

1

If your jobs array gets larger, then you may just have too many requests in flight at the same time. It could be:

  1. Because you run out of socket resources locally
  2. You overload the host you're making requests to and its way of dealing with too many requests from one source is to just hang up (this is the most likely explanation that matches the error you get).
  3. It takes you longer than 5 seconds to finishing doing all the requests so you're starting the next iteration before the previous one is done.

I'd suggest the following solution to handle all those issues:

const Promise = require('bluebird');
const utils = require('utils');
client.getAsync = utils.promisify(client.get);

function runJobs() {
    var jobs = ['J1', 'J2', 'J3', 'J4'];
    var full_data = {};
    Promise.map(jobs, function(job) {
        return client.getAsync("MY URL").then(data => {
            io.emit('progressbar', data);
        }).catch(err => {
            console.log('something went wrong on the request', err.request.options);
            // eat the error on purpose to keep going
        });
    }, {concurrency: 5}).then(() => {
        // All done, process all final data here            
        // Then, schedule the next iteration
        setTimeout(runJobs, 5000);
    });
}

runJobs();

This runs a max of 5 requests at a time (you can play with adjusting that number) which solves both items 1 and 2 above. And, instead of setInterval(), it uses a recurring setTimeout() so that it won't ever schedule the next iteration until the prior one is done (even if the target server gets really slow).

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • @BLang - FYI, I realize the `utils.promisify()` function won't work properly here because `client.get()` doesn't appear to take the right kind of callback that has two parameters `(err, data)` which is needed for the promisify to work properly. I'd need to know a little more about what `client.get()` is to know how to promisify it properly. – jfriend00 Feb 21 '18 at 23:46
0

It turned out to be the client.get() request causing the error. Here is my code to fix this. It still errors, but at least the error is handled and wont cause the node server to crash. If there is a more eloquent way of handling this, please let me know!

setInterval(function(){
  var jobs = ['J1', 'J2', 'J3', 'J4'];
  var full_data = {};
  for(var i = 0; i < jobs.length; i++){
    client.get("MY URL", function (data, response) {
        io.sockets.emit('progressbar', data);
      }).on('error', function (err) {
        console.log('something went wrong on the request', err.request.options);
    });
  }
  console.log(full_data);

}, 5000)
BLang
  • 930
  • 3
  • 16
  • 35