2

I wrote a small program in C# using ssh.net that connects to an SSH server, makes some changes, and restarts the SSHD service on the SSH server.

Unfortunately, the SSH server is being run on a slow embeded system, and the SSHD service is not restarted quickly enough to do so seamlessly (without disconnecting the client). The SSH service actually takes about a minute to restart. This is fine.

My code looks this:

Console.WriteLine("Restarting SSHD service on modem...");

try
{
    using (var client = new SshClient(IPAddress, "root", "PASSWORD"))
    {
        client.ConnectionInfo.Timeout = TimeSpan.FromSeconds(10);
        client.Connect();

        client.RunCommand("service sshd restart");

        client.Disconnect();
    }
}
catch (System.Net.Sockets.SocketException)
{
    //We got disconnected because we just restarted the sshd service
    //This is expected
    Console.WriteLine("\tSuccess");
}
catch
{
    //We got disconnected for some other reason
    Console.WriteLine("\tIt appears there was a problem restarting the sshd service");
}

The issue I'm having is that the ssh client takes up to a minute, or sometimes longer, to figure out that the SSH server is no longer responding, and throw a System.Net.Sockets.SocketException. Is there any way to shorten this timeout? I don't want it to reconnect - I just want it to throw the exception sooner. Would this be some sort of ssh.net specific timeout value, or a System.Net.Sockets timeout?

Tal
  • 322
  • 1
  • 7
  • 15
  • I'm not sure what the remote server, with the Linux box I have "service restart" does not kill existing client connections, have you set TCPKeepAlive or ServerAliveInterval etc. to make sure connection won't timeout due to slow command? I got impression that without keepalive setting the connection will be cut in about 1 minute which matches what you see. – Hang Mar 25 '16 at 04:27
  • 1
    The only relevant-sounding values I have found in my SshClient object is client.KeepAliveInterval and client.ConnectionInfo.RetryAttempts. KeepAliveInterval sounds like another name for TCPKeepAlive, so I tried setting that to 10 seconds, but that had no effect. I also set RetryAttempts to 1, but that had no effect either - it still takes over a minute to throw the exception. Is there another place I can set TCPKeepAlive and ServerAliveInterval? – Tal Mar 25 '16 at 13:38
  • Maybe this one: http://fredericiana.com/2009/10/21/keeping-ssh-from-disconnecting-automatically/ – Hang Mar 25 '16 at 14:01
  • The link deals with configuring the ssh server to send keepalive messages to clients so that the session doesn't drop. I want the session to drop faster. I also can't modify the ssh server's sshd_config, as I am dealing with hundreds of SSH servers, and modifying sshd_config on all of them ahead of time is not feasible. – Tal Mar 25 '16 at 14:37
  • Can you do a manual "service sshd restart" from the server to make sure it does kill the client connection? My point is that the SSH server should not kill the client, it is because of "service restart" takes more than a minutes which means the connection is idle for more than a minute and it got terminated, and thus I suggest tuning the KeepAlive configuration. If you are running service restart on hundreds of SSH servers then change the configuration is about the same thing ("man sed" and look for "-i"). – Hang Mar 25 '16 at 20:29
  • Yes - "service sshd restart" in my case makes the ssh server stop listening for over a minute, and like you said, because the client is idle for so long, it eventually disconnects. Yes - it makes sense to adjust KeepAlive. How would I do this? Also, I don't see how sed would help - sed only works on localhost (you can't make it change a file on 100 ssh servers), and I cannot/will not change the sshd_config on the sshd servers for any reason - my only option is to change my code to compensate for any sshd settings that are already configured. – Tal Mar 25 '16 at 20:56
  • Just want to make it clear, "makes the ssh server stop listening for over a minute", but does it affect existing connection? Let's say you ssh to the host, run service restart, will you be kicked out of the SSH server or you are still connected after the restart? Let's make this part clear (to understand client got disconnected by being idle for too long or by server restarting), then we can try to get a solution or workaround. – Hang Mar 25 '16 at 21:01
  • If I was to use a regular SSH client (linux ssh client, or putty) to connect to the ssh server and issue "service sshd restart", the session will stop responding for about a minute, during which time, I will not be able to enter any commands, and I will not be able to connect to the server with any other client. After a minute, depending on the client, it may resume the session (allow me to type stuff again), or it may time out before then. – Tal Mar 25 '16 at 23:24
  • You can set KeepAliveInterval (like [this](http://stackoverflow.com/questions/27421094/c-sharp-ssh-port-forwarding-with-ssh-net)) so the client check more often thus to disconnect faster if the remote server is not responding, if I read [source](https://github.com/chartek/sshnet/blob/master/Renci.SshNet/BaseClient.cs) correctly), but based on your description, SocketException does not really mean the operation was successful, the best way to me is run service restart in background, then check the service later. – Hang Mar 26 '16 at 00:38
  • As stated in my previous comment, client.KeepAliveInterval had no effect. Also, my program has absolutely nothing else to do, and no reason to wait for the ssh service to start responding again. All I want is to send the "service sshd restart" command to the server, and disconnect as quickly as possible, so I can tell the user of the program that the service is being restarted. – Tal Mar 26 '16 at 04:31
  • 1
    `All I want is to send the "service sshd restart" command to the server, and disconnect as quickly as possible` - you can do `"service sshd restart &"` or, depends on distro on SSH server, maybe `"service sshd restart >/dev/null 2>&1 &"` – Hang Mar 26 '16 at 05:55
  • Tried using `&` a bunch of different ways, but surprisingly, this seems to have worked now - sort of. I don't think I tried getting rid of stdout and stderr before as you do above. Unfortunately, while this results in the client.RunCommand returning very quickly, it then gets hung up on client.Disconnect() for exactly as long (over a minute). If I comment out client.Disconnect(), it gets hung up trying to leave the "using" statement for over a minute instead. I'll see if there's a way to kill the connection right after the client.RunCommand(). – Tal Mar 26 '16 at 12:47

1 Answers1

3

Thanks to Hang's suggestions, the following code has solved the problem:

Console.WriteLine("Restarting SSHD service on modem...");

try
{
    var client = new SshClient(IPAddress, "root", "PASSWORD")
    client.ConnectionInfo.Timeout = TimeSpan.FromSeconds(10);
    client.Connect();

    client.RunCommand("service sshd restart >/dev/null 2>&1 &");
    Console.WriteLine("\tSuccess");
}
catch (System.Net.Sockets.SocketException)
{
    //We got disconnected because we just restarted the sshd service
    //This is expected
    Console.WriteLine("\tSuccess");
}
catch
{
    //We got disconnected for some other reason
    Console.WriteLine("\tIt appears there was a problem restarting the sshd service");
}

The things to note are:

  • The RunCommand line now uses >/dev/null 2>&1 to get rid of stdout and stderr
  • The RunCommand line now uses & to run the command in the background
  • The try block now contains a Console.WriteLine("\tSuccess");, because this is the most likely result of the RunCommand() - the fact that it will keep going instead of throwing any exceptions
  • There is no longer a client.Disconnect() or a using statement, so nothing attempts to disconnect from the ssh server - the code just keeps going

The reason this works in my case is because my program basically does nothing else after this - it just exits. Others may want to wait until the SshClient becomes available again, and disconnect properly to clean up and release the resources.

Tal
  • 322
  • 1
  • 7
  • 15