ClientWebSocket on Linux throws AuthenticationException (SSL)

Question

I run the following websocket client code on windows and everything works fine - like expected. But if the code is published for linux-arm and copied to a RaspberryPi3 (runs under Raspian) it will end up in an AuthenticationException.

csproj file content:

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="10.0.3" />
    <PackageReference Include="System.Net.WebSockets.Client" Version="4.3.1" />
  </ItemGroup>

The connection attempt: (the point where the exception is thrown)

private readonly ClientWebSocket _socket;

public ApiConnection()
{
    _socket = new ClientWebSocket();
}

public async Task Connect()
{
    // the uri is like: wss://example.com/ws
    await _socket.ConnectAsync(new Uri(_settings.WebSocketUrl), CancellationToken.None);

    if (_socket.State == WebSocketState.Open)
        Console.WriteLine("connected.");
}

Exception stack:

System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
         at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
         at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
         at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
         at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
         at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Net.WebSockets.WebSocketHandle.<ConnectAsyncCore>d__24.MoveNext()
         at System.Net.WebSockets.WebSocketHandle.<ConnectAsyncCore>d__24.MoveNext()
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Net.WebSockets.ClientWebSocket.<ConnectAsyncCore>d__16.MoveNext()
      --- End of stack trace from previous location where exception was thrown ---
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.GetResult()

The target websocket server is running behind a nginx proxy on Ubuntu. I think the problem relies on the client because if the code is executed on windows everything works fine.

I tried also importing the CA certifacte into Raspians "certificate store". With no luck.

UPDATE:
A http connection (ws://) works also on linux. It seems, the WebSocketClient didn't trust my LetsEncrypt cert?

Your client is rejecting the certificate from the Server. One suggestion would be to add a certificate call back and log the certificate error details. This is likely to give more details about WHY the certificate is getting rejected. — Subbu, Aug 18 '17 at 03:46
@Subbu What's the best way to register such a callback? The `ServicePointManager` is not available on .NET Core — senz, Aug 18 '17 at 04:28
@senz LetsEncrypt isn't trusted by your device, such free certification only works on major browsers. This has nothing to do with windows or linux. You will have to install LetsEncrypt's ROOT CA in your certificate store. — Akash Kava, Aug 24 '17 at 07:51

score 1 · Answer 1 · answered Aug 21 '17 at 05:55

The certificates that are validated on windows, won't necessarily validate on Linux. Each of the operating systems are using different certificates and different methods to validate them, furthermore there are certificates known to Linux which are not supported by windows.

There can be a situation, where your LetsEncrypt cert is recognized by windows but Linux did not recognize this and thus, threw and exception of AuthenticationException stating clearly

"The remote certificate is invalid according to the validation procedure"

Meaning the Linux tried to validate the certificate, but failed, as it was not recognized to the Linux at all but your windows recognized it and acted as expected.

I don't know much as to which certificates will work on which Linux, but I would recommend to research this thing in order to find a way to use a certificate, that both the windows and Linux can recognize, validate and work with.

In case of a HttpClient the certifacte gets validated. So the linux recognition should be fine? Could this error rely on `_clientWebSocket.Options.ClientCertificate`? This is default `null` but when i create an empty X509 certificate, the error message in my questions turns into a `NullReferenceException`. — senz, Aug 22 '17 at 04:43
The error was the client web socket from the beginning, windows and Linux are validating them all different while HttpCilent is validated all the same, there is no difference between Http call to Linux or windows, but there is a difference between the sockets and how it works in Linux and windows. — Barr J, Aug 22 '17 at 05:09

score 1 · Answer 2 · answered Aug 21 '17 at 06:50

I tried something similar recently (although I used Mono instead of .Net Core), and in my case it was simply that the system time on the Raspberry was off by a couple of days(!), thereby running outside the certificate's "Valid From" to "Valid To" timestamps. This can happen if the Raspi has no internet connection to synch it's time via NTP. Raspberries do not contain a hardware clock with a buffer battery, so they lose track of time when not powered on.

The first step would be to log onto the Pi and run date, to see if the system clock is correct.

IF this is your problem, you have several possible fixes:

Enable internet access to the Raspberry Pi, so it can synch its clock via NTP
Set the correct time yourself, then keep the Raspi powered on at all times, making sure to manually set the time whenever you power it on
Install a hardware clock ($3 - $5, about ten minutes worth of work if you follow instructions), and be done with the problem (as long as the battery lasts)

Another idea could be to check where .Net Core expects the CA-certs to be installed. At least using Mono this differs from the Linux defaults. I used the X509Store C# API to install the certificate instead of the (Debian-)Linux system tools.

If it is a workplace, he cannot use the third option unfortunately unless he can install components on company products, which is not very likely... — Barr J, Aug 21 '17 at 11:41
The clock on the pi works fine . In use case of a `HttpClient` the certifacte works. So it seems not to be a location problem. — senz, Aug 22 '17 at 04:44

score 1 · Accepted Answer · answered Aug 24 '17 at 09:46

This happens when the browser / client does not trust the SSL cert that the server is throwing at it.

To test, load the same url / a url on the same site in your browser, you should get a warning.

When the cert issue is resolved the warning will go away.

The exact process for resolving SSL cert issues is dependent on a lot of things like ...

OS, Web Server, Cert Authority, Cert Providers Portal so it's near impossible for anyone on here to give you specifics about fixing cert issues, but that said ...

There is however a bit of generic advice on this here on the SE network ...

https://unix.stackexchange.com/questions/90450/adding-a-self-signed-certificate-to-the-trusted-list

https://unix.stackexchange.com/questions/17748/trust-a-self-signed-pem-certificate

In your case, as rasbian is based on debian, some standard debian advice might help ...

In Debian, the certificates stash is located in /etc/ssl/certs/. This directory contains by default a series of symlinks that points to the certificates installed by the ca-certificates package (including the needed symlinks generated by c_rehash(1)) and a ca-certificates.crt which is a concatenation of all these certificates. Everything managed by the update-ca-certificates(8) command which is taking care of updating the symlinks and the ca-certificates.crt file.

Adding a new (CA) certificate to the stash is quite easy as update-ca-certificates(8) is also looking for files in /usr/local/share/ca-certificates/, the administrator just has to place the new certificate in the PEM format in this directory (with the .crt extension) and run update-ca-certificates(8) as root. All the applications on the system (wget, …) should now trust it.

The other possible solution might be "I trust my code to not request bad url's so i'll ignore SSL cert errors" which you could do with something like this ...

C# Ignore certificate errors?

... but that's not ideal, at least it gives you a work around until you can resolve the issue, worst case you could still check but by coding your own check instead of just a blanket return true.

final point:

I often find no matter what the OS, doing something as simple as a reboot or two in between tests / checks can clear something out that you wouldn't normally consider to be an issue.

ClientWebSocket on Linux throws AuthenticationException (SSL)

3 Answers3