Console APP migrated from .net core 2.2 to .net core 3.1 stopped allowing external HTTPs calls

Question

I have a console APP which is has a hosted service packed up in a docker container in netcore 2.2:

FROM microsoft/dotnet:2.2-sdk AS builder
WORKDIR /service
# copy csproj and restore as distinct layers
COPY . .
RUN dotnet restore ./Project.sln
# copy everything else and build
COPY . .
RUN dotnet publish ./Project/Project.csproj -c Release -o /service/out

# build runtime image
FROM microsoft/dotnet:2.2.0-aspnetcore-runtime
WORKDIR /service
COPY --from=builder /service/out ./
ENV ASPNETCORE_ENVIRONMENT Production
ENTRYPOINT ["dotnet", "Project.dll"]

I have been using RestSharp 106.6 with no issues. Then today I decided to migrate to net core 3.1 so apart from changing the assembly version in all projects with success and running the console app successfully as well, I modified my Dockerfile:

FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS builder
WORKDIR /service
# copy csproj and restore as distinct layers
COPY . .
RUN dotnet restore ./Project.sln
# copy everything else and build
COPY . .
RUN dotnet publish ./Project/Project.csproj -c Release -o /service/out

# build runtime image
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS runtime
WORKDIR /service
COPY --from=builder /service/out ./
ENV ASPNETCORE_ENVIRONMENT Production
ENTRYPOINT ["dotnet", "Project.dll"]

Then I started getting the following error when using the restsharp client:

The SSL connection could not be established, see inner exception. Authentication failed

But the inner exception is saying the same. My guess is that there is something missing in the docker image that is preventing the client to work because it works locally - I don't think there is some configuration missing otherwise the local version would not work either.

Have you guys come across this problem before?

Thanks

UPDATE

I'm logging out the actual exception message that I get when using RestSharp:

 ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
 ---> Interop+OpenSsl+SslException: SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.
 ---> Interop+Crypto+OpenSslCryptographicException: error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol
   --- End of inner exception stack trace ---
   at Interop.OpenSsl.DoSslHandshake(SafeSslHandle context, Byte[] recvBuf, Int32 recvOffset, Int32 recvCount, Byte[]& sendBuf, Int32& sendCount)
   at System.Net.Security.SslStreamPal.HandshakeInternal(SafeFreeCredentials credential, SafeDeleteContext& context, ArraySegment`1 inputBuffer, Byte[]& outputBuffer, SslAuthenticationOptions sslAuthenticationOptions)
   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ReadFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Security.SslStream.ThrowIfExceptional()
   at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at System.Net.HttpWebRequest.SendRequest()
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at RestSharp.Http.GetRawResponseAsync(IAsyncResult result, Action`1 callback)
   at RestSharp.Http.ResponseCallback(IAsyncResult result, Action`1 callback)

That's just me rolling forward again the project using .NET Core 3.1. If that stays in .NET Core 2.2 it just works. The code to perform the HTTP Call:

public async Task<MyResponse> GetAsync()
        {
            MyResponse myResponse = null;

            var stopWatch = new Stopwatch();

            stopWatch.Start();

            try
            {
                var request = new RestRequest(Method.GET);

                request.AddHeader("Accept", "application/json");

                var restResponse = await _restClient.ExecuteGetAsync(request);

                if(restResponse.IsSuccessful)
                {
                    gobResponse = JsonConvert.DeserializeObject<MyResponse>(restResponse.Content, DateTimeConverter);

                    Logger.Info($"Retrieved my response. Took: {stopWatch.ElapsedMilliseconds}ms. StatusCode={restResponse.StatusCode.ToString()}");
                }
                else
                {
                    Logger.Error(restResponse.ErrorException.ToString(), "Error performing request");
                }
            }
            catch(Exception ex)
            {
                Logger.Error(ex.ToString(), $"Error executing HTTP client");
            }

            return myResponse;                
        }

The error has nothing to do with .NET Core 3, it complains about SSL and authentication. The question doesn't contain any code, so it's impossible to say what's wrong or what the code does. Are you trying to use SSL with an invalid or self-signed certificate perhaps?. Even the exception text is missing. Post the *full* text, not just the message. The message clearly says that the details are in the inner exception. You can easily get the full text with `Exception.ToString()` — Panagiotis Kanavos, Jan 24 '20 at 13:41
OK re the code, I will add it however the question is still valid. How come an upgrade of the .net core version could affect that? That's the actual question. No code changes, just upgrading the version in the project files + changes in the dockerfile — Carlos Torrecillas, Jan 24 '20 at 13:49
Or perhaps the certificate invalid? Did it expire perhaps? Or were you using code to disable certificate validation that no longer works? You're asking people to guess what your code does, how you use SSL. This isn't about Docker so the information posted here isn't helping at all. The *only* relevant thing is that single line from the exception text — Panagiotis Kanavos, Jan 24 '20 at 13:56
just pasted both the exception and the code that executes the request. Same code, just version change. One works, the other fails. And both work on my local machine but when I package them up in a container that's the result I get on the 3.1 version — Carlos Torrecillas, Jan 24 '20 at 14:14
It's definitely *not* same - `SSL Handshake failed with OpenSSL error - SSL`. So it's Linux which means OpenSSL, another dependency, is actually used. That *external* dependency says `error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol`. Are you calling a service that *doesn't* use TLS1.2 perhaps? If you google that error `error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol` you'll see that various Linux distributions now require TLS1.2 by default. The OpenSSL version in your Docker image was probably upgraded recently — Panagiotis Kanavos, Jan 24 '20 at 14:25
Does this answer your question? [OpenSSL v1.1.1 ssl\_choose\_client\_version unsupported protocol](https://stackoverflow.com/questions/53058362/openssl-v1-1-1-ssl-choose-client-version-unsupported-protocol) — Panagiotis Kanavos, Jan 24 '20 at 14:25
The preferred solution would be to upgrade the service you call to TLS1.2, or contact whoever owns it to modify it so it *does* use TLS1.2. Major services like banks, airlines etc switched to TLS1.2 as a minimum 4 years ago. Cloud providers did the same a couple of years ago. There's no reason to have a service that uses anything less than TLS1.2 — Panagiotis Kanavos, Jan 24 '20 at 14:27
The suboptimal solution as the possible duplicate shows, is to configure OpenSSL to accept lower TLS versions. That's *definitely* not a good idea though and will have to be disabled sooner or later — Panagiotis Kanavos, Jan 24 '20 at 14:28
Or perhaps the remote service started using a lower TLS version by mistake? — Panagiotis Kanavos, Jan 24 '20 at 14:30
Just to be clear. I'm targeting the same endpoint (HTTPS). I'm using the same hosting provider. I'm using the same code. When I use v.2.2 it works. When I use v3.1 it fails. If I run both 2.2 and 3.1 on my machine it works. I think it has to be either an incompatibility between RestSharp and NC3.1 or the image itself of the .net core 3.1 has something different in terms of OS TLS configuration — Carlos Torrecillas, Jan 24 '20 at 14:31
The error you posted clearly shows it's *OpenSSL* that complains. And if you just google for the *relevant* error, you'll see it's caused because **now** both OpenSSL and the distros require TLS1.2. Your hosting provider doesn't matter, what service are you calling? What does *it* use? That's not something that's configured by the hoster. — Panagiotis Kanavos, Jan 24 '20 at 14:38
You can see what TLS version is used simply by browsing to the service URL and clicking on the shield icon at the left of your browser's address bar. A typical way to troubleshoot such issues is to use a debugging proxy like Fiddler and inspect the requests and responses, including the tunneling calls — Panagiotis Kanavos, Jan 24 '20 at 14:40
I think you have pointed me to the right direction. I got Postman and went to the Protocols disabled during handshake. Disabling TLS v1 it then fails. What I am assuming is that the new .NET Core 3.1 image sets by default another TLS version or disables TLS v1 therefore it fails — Carlos Torrecillas, Jan 24 '20 at 14:46
No it doesn't. The duplicate question explains that the *distros and OpenSSL do so*. 2.2 was released over a year ago. The *new* images contain an entire year's worth of Linux fixes. Actually, you're lucky - TLS v1 is considered broken. TLS 1.1 is the minimum. Had anything happened, you'd be liable. For example [PCI "suggested" organizations move away from TLS 1](https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_1.0) for credit card payments with a December 2018 deadline. — Panagiotis Kanavos, Jan 24 '20 at 14:53
I understand. Thank you. You'd be surprised but that endpoint I'm consuming is from the Spanish government...perhaps I should get in touch with them. In any case I got it working by adding the ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls; on my Program.cs — Carlos Torrecillas, Jan 24 '20 at 15:04
In that case you won't even be able to visit it with a web browser in 1 month's time. The link I posted shows that all browsers will discontinue TLS1.0 by March — Panagiotis Kanavos, Jan 24 '20 at 15:05
As for governments, even the Greek one disallowed TLS1.1 a couple of years ago. You can expect the Spanish agency responsible (let's not call it a snafu yet) for that misconfiguration to scramble to fix it once people realize it's now inaccessible. — Panagiotis Kanavos, Jan 24 '20 at 15:06

score 8 · Answer 1 · answered May 12 '20 at 00:50

8

You can try downgrading the TLS version from 1.2 to 1.0 by adding this line into your Dockerfile (runtime image)

RUN sed -i "s|TLSv1.2|TLSv1.0|g" /etc/ssl/openssl.cnf

answered May 12 '20 at 00:50

hkarask

423
5
9

That is not a bad idea. Thanks for the tip – Carlos Torrecillas May 20 '20 at 21:20
That solved my problem. Good job just for reference, I had AWS RDS Mysql and docker container .net app 3.1 running in Kubernetes cluster. – Boncho Valkov Dec 01 '20 at 09:45
Unfortunately, this does not seem to work for .net5 base images which ship `OpenSSL 1.1.1d 10 Sep 2019` if we can trust `openssl version`. – mbx Dec 02 '20 at 10:09

mbx · Answer 2 · 2020-12-16T07:30:48.260

3

When using net5 and a recent Data.SqlClient lib, only setting TLS to 1.0 can cause a deadlock when executing a query.

# downgrade min TLS version to support older unpatched SQL Server (i.e. SQL Server 2008R2)
# you should patch those machines to support TLSv1.2 though
# see: https://support.microsoft.com/en-us/help/3135244/kb3135244-tls-1-2-support-for-microsoft-sql-server
RUN sed -i "s|MinProtocol = TLSv1.2|MinProtocol = TLSv1|g" /etc/ssl/openssl.cnf && \
    sed -i 's|CipherString = DEFAULT@SECLEVEL=2|CipherString = DEFAULT@SECLEVEL=1|g' /etc/ssl/openssl.cnf

While this workaround probably is what you need immediately, applying the fix for SQL Server is what you (or your ops team) should do in the near future.

edited Dec 16 '20 at 07:30

answered Dec 02 '20 at 13:06

mbx

6,292
6
58
91

Thanks for this. I don't have a SQL DB so that should be OK anyway – Carlos Torrecillas Dec 02 '20 at 13:28
@CarlosTorrecillas I recently faced the same issue with an old unpatched 2008R2 so I wanted others to know how to get it to work again and how to fix it forever as this question shares the same underlying issue and ranks in google.If you don't connect to SQL server, that's of course a non-issue for you. – mbx Dec 02 '20 at 14:23
Nice one. Thanks for this! – Carlos Torrecillas Dec 02 '20 at 14:24
@mbx do you have a sample docker file where you used this? – Greg Bala Apr 29 '21 at 19:46
1

@GregBalajewicz The one I'm thinking of is not public (and afair no longer in production as the patch was applied and shortly after the 2008R2 was phased out), it basically boiled down to [this](https://pastebin.com/FqRV23Bs) – mbx May 04 '21 at 06:20

Console APP migrated from .net core 2.2 to .net core 3.1 stopped allowing external HTTPs calls

2 Answers2

Linked