0

I'm porting LXD.NET to .Net Standard 2.0. REST APIs via HTTPS works well with .Net Core and .Net Framework.
But using WebSocket APIs (e.g. 1.0/containers/<name>/exec ), ClientWebSocket throws Authentication Exception on only .Net Core. ( On .Net Framework, it doesn't throw and works well. )

LXD is running on Ubuntu 18.04. I tested in two clients, one is running on Windows 10, the other is running on Linux (the same linux comupter runnning LXD).

All source code is Here ( https://github.com/GnicoJP/lxd-dotnet-websocket-test ).

Step to reproduce(Server side)

LXD setting

$ sudo apt install lxc lxd  
$ sudo lxd init

I configured LXD to be available over the network. 

$ sudo lxc launch images:alpine/3.8 &lt;container_name&gt;

I launched an image.

SSL Key/Certificates generating

$ openssl genrsa 2048 > client.key  
$ openssl req -new -key client.key > client.csr  
$ openssl x509 -days 3650 -req -signkey client.key < client.csr > client.crt  
$ openssl pkcs12 -export -in client.crt -inkey client.key -out client.p12

Bring client.p12 to Client PCs.

I Registered SSL Certificate 

$ sudo lxc config trust add client.key

Source Codes

Entry Point is Here. I'm calling exec API.(wait-for-websocket is true)
Parsing Exec API's result is Here. I think WebSocket's URI is not wrong.
Opening ClientWebSocket is Here. On .Net Framework, WebSocket works well with/without certificates.

Thrown Exceptions

System.AggregateException
HResult=0x80131500
Message=One or more errors occurred. (Unable to connect to the remote server)
Source=System.Private.CoreLib
Stack Trace:
    at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) in Task.cs:line 2840
    at System.Threading.Tasks.Task.Wait() in Task.cs:line 2706
    at lxd_dotnet_websocket_test.Program.Main(String[] args) in lxd-dotnet-websocket-test\Program.cs:line 20

Inner Exception 1:
System.Net.WebSockets.WebSocketException: Unable to connect to the remote server
HResult=-2147467259
Stack Trace:
    at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
    at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken)
    at LXD.ClientWebSocketExtensions.CreateAndConnectAsync(String url, API API) in .\libs\LXD\Util\ClientWebSocketExtensions.cs:line 19
    at LXD.Domain.Container.ContainerExecResult.ContainerExecResultWithWebSockets.TestCreate(API API, ContainerExec exec, JToken response, String operationUrl) in .\libs\LXD\Domain\Container.cs:line 228

Inner Exception 2:
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
HResult=-2146233087
Stack Trace:
    at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
    at System.Threading.Tasks.ValueTask`1.get_Result()
    at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    at System.Threading.Tasks.ValueTask`1.get_Result()
    at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
    at System.Threading.Tasks.ValueTask`1.get_Result()
    at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)

Inner Exception 3:
AuthenticationException: The remote certificate is invalid according to the validation procedure.
HResult=-2146233087
Stack Trace:
    at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
    at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
    at System.Net.Security.SslState.ThrowIfExceptional()
    at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
    at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
    at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
    at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__47_1(IAsyncResult iar)
    at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
    at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)

I'm begginer in StackOverflow. Please tell me if my question or my English is not good.

Gnico
  • 1
  • 4
  • Are you always running with same machine (and same account) as the Server? It appears from the error message the "remote certificate is invalid". Are you running the service with ADMIN privileges? I usually recommend using a sniffer like wireshark or fiddler and comparing the HTTP headers in the work application and non working application. HTTP uses TCP as the transport layer. A TCP datagram has maximum size of ~1500 bytes. So an HTTP consists of one or more TCP messages. If TLS is failing it would show up on the TCP datagrams. In TCP [FIN] indicates a conneciton closing. – jdweng Feb 09 '19 at 10:39
  • Thanks for your reply. I'm not running with same machine. but, If I compile it with .Net Framework, it works well (even if I access from different machine.). I want to know why accessing WebSocket APIs with .Net Core fails. I'll try to use Wireshark, thanks. – Gnico Feb 09 '19 at 11:39
  • LXD is running with admin privileges. I don't think it is occurred by privileges mistakes because other program created with other libraries(pylxd, and more) would be unable to access websocket same as this built with .NetCore. – Gnico Feb 09 '19 at 11:55
  • I don't thinki the issue is with the code. I believe the issue is the server. It is possible the version of Net Core is different on the bad server. The Net library in a lot of cases are wrappers calling windows dlls. So if the windows libraries are different or the Net libraries are different (on different machines) the code doesn't always work. That is why I recommend publishing application and installing on deploy machines which updates windows dlls. In this case the error message indicates that something is wrong with a certificate. The certificate date could of expired. – jdweng Feb 09 '19 at 12:40
  • Programs built with .Net Core and .Net Framework are running on the same Computer, same Windows Environment, same certificate. (If previous comment, " I'm not running with same machine. " makes you confused, I'm sorry. It means Server and Client are different machine.) If the certificate is invalid, I wonder .Net Framework doesn't throw Exception. I'm sorry for not knowing, and Thanks for reply. – Gnico Feb 09 '19 at 13:24
  • These issues are hard to debug through postings because there are many different reasons for failures.Best I can do is help teach what to look for. Eventually the OPs find the solutions themselves. I've seen issues when TLS options in the Internet Explorer settings were not set properly.People don't realize the IE settings are used by Net.They are in the users temporary IE folder. Don't know if Core and Net applications are using same http headers. Negotiation between client and server are performed using headers to get transfer mode. If the request is different the mode can be different. – jdweng Feb 09 '19 at 14:40
  • Oh...it's bad news... I also wonder why REST APIs via HTTPS (not WebSocket) succeed in authentication and GET/POST jsons correctly. If something wrong have been in certificates or IE settings, REST APIs via HTTPS would fail, I think. Is this understanding mistaken? I tried Wireshark. It is interesting that .Net Core does TLS Session Resumption but Net Framework doesn't. Net Core reuses Client key used by past REST APIs, but Net Core sends Encrypted Alert to the Server soon. Next TCP connection was reset, and recreate session ticket, but soon Server sends Encrypted Alert. What's happen...? – Gnico Feb 10 '19 at 01:44
  • So I'm trying to get logs from .Net Core's EventSources to get what .Net Core is doing. Thanks for your advice and reply. – Gnico Feb 10 '19 at 02:03
  • I solved. I got logs [by using PerfView](https://github.com/dotnet/corefx/blob/master/Documentation/debugging/windows-instructions.md#using-perfview). It says "Certificate Chain was processed, but terminated because it was a not-trusted root certificate.". So I added LXD's certificate to trusted root certificates by certmgr, then .Net Core stopped throwing Exception. It seems that .Net Core checks whether it is trusted root certificate, but .Net Framework doesn't. Thanks for your suggestion, I'm able to solve this problem. – Gnico Feb 10 '19 at 06:49
  • Like I said, the OP always finds the issue themselves. – jdweng Feb 10 '19 at 11:06

1 Answers1

0

.Net Framework doesn't checks whether the certificate used in WebSocket is trusted, but .Net Core does. I was unable to get how to solve from Exception, so I investigated by using PerfView.

How to Solve

I installed LXD's Server Certificates. Bring server.crt file(/var/lxd/server.crt) to Client PCs, then install as trusted root Certificate.
Please read how to install: Windows, Linux

Community
  • 1
  • 1
Gnico
  • 1
  • 4