This sort of authentication is typically done using the Windows SSPI API, which wraps the various security service providers ala NTLM / Kerberos.
Callers of SSPI can obtain a handle to the current user's credentials and use this credential handle to ask a SSP to provide authentication tokens. The client sends tokens to the server, the server accepts those and sends tokens to the client, rinse repeat until both sides are happy.
As for the application developer, all you ever have to do is ask for these tokens, ferry them across your connection, and pump them into the appropriate SSPI call.
So typically:
- Client calls AcquireCredentialsHandle()
- Client calls InitializeSecurityContext(), providing no input tokens, returning an output token (an opaque byte array).
- Client sends those tokens to the server.
- Server calls AcceptSecurityContext(), providing the clients tokens as input, returning an output token.
- Server sends his tokens to client.
.. repeats until both functions indicate happy.
I posted the following question to learn how to do this directly: Client-server authentication - using SSPI?
Fortunately, I have a complete .Net wrapper for SSPI, along with some demo programs: NSspi.
Now, I'm not sure on exactly how this integrates into the SSH protocol, or the SSH server that you're using. From my understanding, SSH sessions start by advertising each side's supported authentication mechanism, and I'd estimate that the server is advertising a particular auth provider that is tied directly to SSPI, or maybe kerberos, and just expects you to perform the client side of the above SSPI authentication cycle. The trick would be figuring exactly what the server expects, and how to register a provider with your SSH client library that meets that expectation.