Why does my code disconnects a Socket.io connection in Node?

Question

First of sorry I'am not native english and thanks for interest! I don't know why when I perform some semi-heavy operations on arrays my browser break connection with Socket.io. Browser receives disconnect event, then it reconnects but shouldn't disconnect in the first place.

The code contains a couple nested loops, I know it's not the best practice but I did't figure out any other way of doing the task. I started testing on large arrays of users (about 10k) and the problem started occurring, so I guess it's a performance issue.

I was trying to find memory leeks but with no success, its almost like for the time of performing operations app freezes and breaks Socket.io.

Thank you for trying to help.

var pushUsersToCorrectLeagues = function(appConfig, callback, usersArray, newSeason, db) {
    //determinate which league is the last one
    var leaguesOrderArray = [];
    _.each(newSeason.leagues, function(league) {
        leaguesOrderArray.push(league.key);
        return false;
    });

    var lastLeagueKey = _.last(leaguesOrderArray.sort(function(a, b) {
        return a - b;
    }));

    _.each(usersArray, function(user) {
        var keyUserData = {};
        keyUserData.userLogin = user.userLogin;
        keyUserData._id = user._id;
        keyUserData.avatarUrl = user.avatarUrl;
        keyUserData.isActive = true;
        keyUserData.scores = {};

        //now put users in correct leagues
        var userPushed = false;
        var endedInLeague = null;


        //if found a user in league
        _.each(newSeason.leagues, function(league) {
            _.find(league.users, function(alredyInLeagueUser, i) {
                if (alredyInLeagueUser.userLogin == user.userLogin) {
                    newSeason.leagues[league.key].users[i] = keyUserData;
                    userPushed = true;
                    endedInLeague = league.key;
                    return true;
                }
            });

            return false;
        });

        //user wasnt found in any league, has to be pushed to the last one then
        if (!userPushed) {
            newSeason.leagues[lastLeagueKey].users.push(keyUserData);
            endedInLeague = lastLeagueKey;
        }

        if (endedInLeague) {
            process.nextTick(function() {
                var callback = function(updatedUser) {
                    //console.log('updatedUser', updatedUser);
                    return false;
                };
                updateUserCurrentLeague(callback, user._id, endedInLeague, db);
            });
        }

        return false;
    });

    callback(usersArray, newSeason);
    return false;
};

How long do your time intensive operations last? The client and server exchange heartbeat packets and if one end is busy for a very long time and not responding, then perhaps one end thinks it has disconnected because the heartbeat has not been processed? — jfriend00, Nov 04 '14 at 23:55
It took 6 seconds on the current settings with about 10k users. I lowered heartbeat timeout settings to 5 second I guess this is the problem. How can I fix it? — youbetternot, Nov 05 '14 at 00:13
And 6 seconds seems to be way to much, I guess I need to find a better way to do this task, do you have any suggestions? — youbetternot, Nov 05 '14 at 00:22
`io.set('heartbeat interval', X); io.set('heartbeat timeout', Y);` — xShirase, Nov 05 '14 at 00:22
xShirase I know, but I lowered the timeout on purpose and don't want to change it now — youbetternot, Nov 05 '14 at 00:24
Yes it fixed it but I lowered the timeout because it fixed an issue with not interacting with the app and sleeping a browser/device — youbetternot, Nov 05 '14 at 00:31
For many operations a heartbeat of 30 seconds should be just fine (it depends upon exactly what you're doing though). Connections won't normally just drop. — jfriend00, Nov 05 '14 at 00:31

score 2 · Accepted Answer · edited May 23 '17 at 12:07

It sounds like your operation is taking longer than the socket heartbeat interval so one end of the connection thinks the connection has been dropped.

You have the following choices:

You can lengthen the heartbeat interval on the socket (really just a bandaid, not the best fix).
You can offload your time consuming node operation to a child process and let it run, collect the results and send the results back in JSON. This allows your main node process to stay responsive the whole time.
You can break your time consuming operation up into chunks where each chunk takes no longer than 500ms to run and the server will then be able to process other events while the time consuming operation is going on. This requires reworking how you process things because you can't use .each() loops. Instead, you have to store some state somewhere, use counters, do a certain amount of work, update the state, set a timer to do some other work after other events have been processed, do another chunk of work, repeat until all work done.

FYI, here's an example of how a very large array can be processed in chunks: Best way to iterate over an array without blocking the UI. This was coded for the browser, but the concept is the same in node.

Thanks, this looks like great answer! The issue is like you said caused by heartbeat timeout. I'll try it option two and three tomorrow. Big thanks! — youbetternot, Nov 05 '14 at 00:37

Why does my code disconnects a Socket.io connection in Node?

1 Answers1