2

I'm building a node.js app which in production will act as a SSH client to many servers, some of which may be inaccessible at any given time. I'm trying to write a function which attempts to run a SSH command with each client in its config upon startup, and I'm not able to handle both successful sessions and those which end in error. I wrapped a ssh2 client in a promise. If I remove the third (trash) server and only successes result, this works fine! See the output:

STDOUT: Hello World

STDOUT: Hello World

Session closed
Session closed
Successful session: Hello World,Hello World

But if one of the connections times out, even though I handle the error, I don't get to keep any of my data. It looks like the error message overwrites all of the resolved promises

Successful session: Error: Timed out while waiting for handshake,Error: 
Timed out while waiting for handshake,Error: Timed out while waiting 
for handshake

Here's my code, forgive me if this is a bit scattered, as I've combined a few of my modules for the sake of this question. My goal is to keep the data from the successful session and gracefully handle the failure.

var Client = require('ssh2').Client;

 const labs = {
    "ny1": "192.168.1.2",
    "ny2": "192.168.1.3",
    "ny3": "1.1.1.1"
};

function checkLabs() {
    let numLabs = Object.keys(labs).length;
    let promises = [];

    for(i=0;i<numLabs;i++){
        let labName = Object.keys(labs)[i];
        promises.push(asyncSSH("echo 'Hello World'", labs[labName]));
    }

    Promise.all(promises.map(p => p.catch(e => e)))
        .then(results => console.log("Successful session: " + results))
        .catch(e => console.log("Error! " + e));
}

var sendSSH = function (command, dest, callback) {
var conn = new Client();

        conn.on('ready', function() {
            return conn.exec(command, function(err, stream) {
                if (err) throw err;
                stream.on('data', function(data) {
                    callback(null, data);
                    console.log('STDOUT: ' + data);
                }).stderr.on('data', function(data){
                    callback(err);
                    console.log('STDERR: ' + data);
                }).on('close', function(err) {
                    if(err) {
                        console.log('Session closed due to error');
                    } else {
                        console.log('Session closed');
                    }
                });
                stream.end('ls -l\nexit\n');
            });
        }).on('error', function(err){
            callback(err);
        }).connect({
            host: dest,
            port: 22,
            username: 'root',
            readyTimeout: 10000,
            privateKey: require('fs').readFileSync('link-to-my-key')
        });
};

function asyncSSH(command, dest) {
    return new Promise(function(resolve, reject){
        sendSSH(command, dest, function(err,data) {
            if (!err) {
                resolve(data);
            } else {
                reject(err);
            }
        });
    });
}

checklabs();

How can I better use this promise wrapper to handle whatever errors come from the ssh2 client? Any tips are appreciated.

  • It seems the variable conn comes out of nowhere and just magically exist. Not sure why you define `labs` instead of `const labs = {...` – HMR Jan 11 '18 at 03:44
  • You can only resolve or reject a promise once yet you resolve several times because on data happens several times. This would not cause an error but any call to resolve after the first one will be ignored. Instead you should resolve on close and on data save whatever data you got in an array so you can resolve with that array on close. This does not look like any code you can actually run and demonstrate your problem though. – HMR Jan 11 '18 at 04:02
  • @HMR this is very helpful thanks. I used the define method in a separate file to act as a config, maybe I should just call it a const. Also realizing this would be more useful as a demo if I removed the ssh2 code and demonstrated success/err in psuedo code. I guess handling ssh2 behavior is so central to my problem I need to spend more time with that part. – user9201592 Jan 11 '18 at 04:18
  • To take the problem a little higher level, my issue here is I need to make several transactions using this same piece of ssh2 code. Each one is closed, either successfully in which case the `.on('data'` API results in a `close` or `.on('error'` in the case of a failed connection. Each transaction may take up to 10 seconds with the timeout. I don't care whether they happen synchronously or asynchronously, I just need to save the data from each, and gracefully handle any errors that occur, resuming any remaining ssh sessions. – user9201592 Jan 11 '18 at 04:25
  • Seems a bit odd that you should be getting as far as "Hello World,Hello World" with a single instance of `Client()`. I would have thought you needed either (a) one instance per host, or (b) to perform the connections in series. What happens if the echoed messages include something unique, eg the `ny1`/`ny2` key. – Roamer-1888 Jan 12 '18 at 03:17
  • My mistake was instantiating the conn() object outside of the function sendSSH() - each async call to the promise was trying to connect using the same SSH session, obviously not right. Code updated to reflect the fix. Thanks for your help. – user9201592 Jan 12 '18 at 03:22
  • That makes much more sense. Now I can show you a better way to write the code. – Roamer-1888 Jan 12 '18 at 03:32
  • Any tips are well appreciated! I'm having trouble returning the stdout in the promise, still working through the code. – user9201592 Jan 12 '18 at 04:31

1 Answers1

2

To get best use from each connection, you can (and arguably should) promisify separately :

  • the instatiation of each Client() instance
  • each instance's conn.exec() method (and any other asynchronous methods as required)

This will allow each instance of Client() to be reused with different commands (though not required in this example).

You should also be sure to disconnect each socket when its job is done, by calling client_.end(). For this, a "disposer pattern" is recommended.

With those points in mind and with a few assumptions, here's what I ended up with :

var Client = require('ssh2').Client;

const labs = {
    "ny1": "192.168.1.2",
    "ny2": "192.168.1.3",
    "ny3": "1.1.1.1"
};

function checkLabs() {
    let promises = Object.keys(labs).map((key) => {
        return withConn(labs[key], (conn) => { 
            return conn.execAsync("echo 'Hello World'")
            .catch((e) => "Error: " + e.message); // catch in order to immunise the whole process against any single failure.
                                                  // and inject an error message into the success path.
        });
    });
    Promise.all(promises)
    .then(results => console.log("Successful session: " + results))
    .catch(e => console.log("Error! " + e.message)); // with individual errors caught above, you should not end up here.
}

// disposer pattern, based on https://stackoverflow.com/a/28915678/3478010
function withConn(dest, work) {
    var conn_;
    return getConnection(dest).then((conn) => {
        conn_ = conn;
        return work(conn);
    }).then(() => {
        if(conn_) {
            conn_.end(); // on success, disconnect the socket (ie dispose of conn_).
        }
    }, () => {
        if(conn_) {
            conn_.end(); // on error, disconnect the socket (ie dispose of conn_).
        }
    });
    // Note: with Bluebird promises, simplify .then(fn,fn) to .finally(fn).
}

function getConnection(dest) {
    return new Promise((resolve, reject) => {
        let conn = promisifyConnection(new Client());
        conn.on('ready', () => {
            resolve(conn);
        })
        .on('error', reject)
        .connect({
            host: dest,
            port: 22,
            username: 'root',
            readyTimeout: 10000,
            privateKey: require('fs').readFileSync('link-to-my-key')
        });
    });
}

function promisifyConnection(conn) {
    conn.execAsync = (command) => { // promisify conn.exec()
        return new Promise((resolve, reject) => {
            conn.exec(command, (err, stream) => {
                if(err) {
                    reject(err);
                } else {
                    let streamSegments = []; // array in which to accumulate streamed data
                    stream.on('close', (err) => {
                        if(err) {
                            reject(err);
                        } else {
                            resolve(streamSegments.join('')); // or whatever is necessary to combine the accumulated stream segments
                        }
                    }).on('data', (data) => {
                        streamSegments.push(data);
                    }).stderr.on('data', function(data) {
                        reject(new Error(data)); // assuming `data` to be String
                    });
                    stream.end('ls -l\nexit\n'); // not sure what this does?
                }
            });
        });
    };
    // ... promisify any further Client methods here ...
    return conn;
}

NOTES:

  • the promisification of conn.exec() includes an assumption that data may be received in a series of segments (eg packets). If this assumption is not valid, then the need for the streamSegments array disappears.
  • getConnection() and promisifyConnection() could be written as one function but with separate function it's easier to see what's going on.
  • getConnection() and promisifyConnection() keep all the messy stuff well away from the application code.
Roamer-1888
  • 19,138
  • 5
  • 33
  • 44
  • this is great, love your design thanks a lot. This also fails to return the streamed data in the promise, still trying to figure that part out. – user9201592 Jan 12 '18 at 05:04
  • Just edited after reading the documentation again. I still have uncertainties about how the stream behaves. That part may need fixing in order to deliver steamed data via the promise. – Roamer-1888 Jan 12 '18 at 05:09
  • @user9201592, any joy yet? – Roamer-1888 Jan 12 '18 at 21:30
  • The only way I am able to return my data with the promise is at `.on('data'` but this is working fine for me in testing. Thanks again for your help! – user9201592 Jan 13 '18 at 16:52
  • Mmm, suggests that the stream's `close` event doesn't fire, at least not for an 'echo' command.. – Roamer-1888 Jan 13 '18 at 18:01
  • What happens if `resolve(streamSegments.join(''));` is executed unconditionally, without testing `err`? – Roamer-1888 Jan 13 '18 at 18:09