javax.net.ssl.SSLException: Read error: ssl=0x9524b800: I/O error during system call, Connection reset by peer

Question

Our clients are starting to see 100s of these "SSLException error - Connection reset by peer" over the last couple of weeks and I can't figure out why

1. We're using Retrofit with okhttp, no special configuration

   public class OkHttpClientProvider implements IOkHttpClientProvider {
   
       OkHttpClient okHttpClient;
   
       public OkHttpClientProvider() {
           this.okHttpClient = createClient();
       }
   
       public OkHttpClient getOkHttpClient() {
           return this.okHttpClient;
       }
   
       private OkHttpClient createClient() {
           return new OkHttpClient();
       }
   }
   

The above client provider is a singleton. The RestAdapter is built using this injected client (we use dagger) -

RestAdapter.Builder restAdapterBuilder = new RestAdapter.Builder()
                                        .setConverter(converter)
                                        .setEndpoint(networkRequestDetails.getServerUrl())
                                        .setClient(new OkClient(okHttpClientProvider.getOkHttpClient()))
                                        .setErrorHandler(new NetworkSynchronousErrorHandler(eventBus))
                                        );

Based on stack overflow solutions what I've found out -

1. The keep alive duration on the server is 180 seconds, OkHttp has a default of 300 seconds

2. The server returns "Connection: close" in its header but the client request sends "Connection: keepAlive"

3. The server supports TLS 1.0 / 1.1 / 1.2 and uses Open SSL

4. Our servers have moved to another hosting provider recently in another geography so I don't know if these are DNS failures or not

5. We've tried tweaking things like keepAlive, reconfigured OpenSSL on the server but for some reason the Android client keeps getting this error

6. It happens immediately without any delay when you try to use the app to post something or pull to refresh (it doesn't even go to network or have a delay before this exception happens which would imply the connection is already broken). But trying it multiple times somehow "fixes it" and we get a success. It happens again later

7. We've invalidated our DNS entries on the server to see if this what caused it but that hasn't helped

8. It mostly happens on LTE but I've seen it on Wifi as well

I don't want to disable keep alive because most modern clients don't do that. Also we're using OkHttp 2.4 and this is a problem on post Ice cream sandwich devices so I'm hoping it should take care of these underlying networking issues. The iOS client also gets these exceptions but close to a 100 times less (iOS client uses AFNetworking 2.0). I'm struggling to find new things to try at this point, any help / ideas?

Update - Adding full stack trace through okhttp

      retrofit.RetrofitError: Read error: ssl=0x9dd07200: I/O error during system call, Connection reset by peer
              at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:390)
              at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240)
              at java.lang.reflect.Proxy.invoke(Proxy.java:397)
              at $Proxy15.getAccessTokenUsingResourceOwnerPasswordCredentials(Unknown Source)
              at com.company.droid.repository.network.NetworkRepository.getAccessTokenUsingResourceOwnerPasswordCredentials(NetworkRepository.java:76)
              at com.company.droid.ui.login.LoginTask.doInBackground(LoginTask.java:88)
              at com.company.droid.ui.login.LoginTask.doInBackground(LoginTask.java:23)
              at android.os.AsyncTask$2.call(AsyncTask.java:292)
              at java.util.concurrent.FutureTask.run(FutureTask.java:237)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
              at java.lang.Thread.run(Thread.java:818)
       Caused by: javax.net.ssl.SSLException: Read error: ssl=0x9dd07200: I/O error during system call, Connection reset by peer
              at com.android.org.conscrypt.NativeCrypto.SSL_read(Native Method)
              at com.android.org.conscrypt.OpenSSLSocketImpl$SSLInputStream.read(OpenSSLSocketImpl.java:699)
              at okio.Okio$2.read(Okio.java:137)
              at okio.AsyncTimeout$2.read(AsyncTimeout.java:211)
              at okio.RealBufferedSource.indexOf(RealBufferedSource.java:306)
              at okio.RealBufferedSource.indexOf(RealBufferedSource.java:300)
              at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:196)
              at com.squareup.okhttp.internal.http.HttpConnection.readResponse(HttpConnection.java:191)
              at com.squareup.okhttp.internal.http.HttpTransport.readResponseHeaders(HttpTransport.java:80)
              at com.squareup.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:917)
              at com.squareup.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:793)
              at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:439)
              at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:384)
              at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:497)
              at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
              at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:25)
              at retrofit.client.UrlConnectionClient.readResponse(UrlConnectionClient.java:73)
              at retrofit.client.UrlConnectionClient.execute(UrlConnectionClient.java:38)
              at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:321)
              at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240)
              at java.lang.reflect.Proxy.invoke(Proxy.java:397)
              at $Proxy15.getAccessTokenUsingResourceOwnerPasswordCredentials(Unknown Source)
              at com.company.droid.repository.network.NetworkRepository.getAccessTokenUsingResourceOwnerPasswordCredentials(NetworkRepository.java:76)
              at com.company.droid.ui.login.LoginTask.doInBackground(LoginTask.java:88)
              at com.company.droid.ui.login.LoginTask.doInBackground(LoginTask.java:23)
              at android.os.AsyncTask$2.call(AsyncTask.java:292)
              at java.util.concurrent.FutureTask.run(FutureTask.java:237)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
              at java.lang.Thread.run(Thread.java:818)
      ]}

Answer 1

Recently I faced the issue while working on some legacy code. After googling I found that the issue is everywhere but without any concrete resolution. I worked on various parts of the exception message and analyzed below.

Analysis:

1. SSLException: exception happened with the SSL (Secure Socket Layer), which is implemented in javax.net.ssl package of the JDK (openJDK/oracleJDK/AndroidSDK)
2. Read error ssl=# I/O error during system call: Error occured while reading from the Secure socket. It happened while using the native system libraries/driver. Please note that all the platforms solaris, Windows etc. have their own socket libraries which is used by the SSL. Windows uses WINSOCK library.
3. Connection reset by peer: This message is reported by the system library (Solaris reports ECONNRESET, Windows reports WSAECONNRESET), that the socket used in the data transfer is no longer usable because an existing connection was forcibly closed by the remote host. One needs to create a new secure path between the host and client

Reason:

Understanding the issue, I try finding the reason behind the connection reset and I came up with below reasons:

• The peer application on the remote host is suddenly stopped, the host is rebooted, the host or remote network interface is disabled, or the remote host uses a hard close.
• This error may also result if a connection was broken due to keep-alive activity detecting a failure while one or more operations are in progress. Operations that were in progress fail with Network dropped connection on reset(On Windows(WSAENETRESET)) and Subsequent operations fail withConnection reset by peer(On Windows(WSAECONNRESET)).
• If the target server is protected by Firewall, which is true in most of the cases, the Time to live (TTL) or timeout associated with the port forcibly closes the idle connection at given timeout. this is something of our interest

Resolution:

1. Events on the server side such as sudden service stop, rebooted, network interface disabled can not be handled by any means.
2. On the server side, Configure firewall for the given port with the higher Time to Live (TTL) or timeout values such as 3600 secs.
3. Clients can "try" keeping the network active to avoid or reduce the Connection reset by peer.
4. Normally on going network traffic keeps the connection alive and problem/exception is not seen frequently. Strong Wifi has least chances of Connection reset by peer.
5. With the mobile networks 2G, 3G and 4G where the packet data delivery is intermittent and dependent on the mobile network availability, it may not reset the TTL timer on the server side and results into the Connection reset by peer.

Here are the terms suggested to set on various forums to resolve the issue

• ConnectionTimeout: Used only at the time out making the connection. If host takes time to connection higher value of this makes the client wait for the connection.
• SoTimeout: Socket timeout-It says the maximum time within which the a data packet is received to consider the connection as active.If no data received within the given time, the connection is assumed as stalled/broken.
• Linger: Upto what time the socket should not be closed when data is queued to be sent and the close socket function is called on the socket.
• TcpNoDelay: Do you want to disable the buffer that holds and accumulates the TCP packets and send them once a threshold is reached? Setting this to true will skip the TCP buffering so that every request is sent immediately. Slowdowns in the network may be caused by an increase in network traffic due to smaller and more frequent packet transmission.

So none of the above parameter helps keeping the network alive and thus ineffective.

I found one setting that may help resolving the issue which is this functions

setKeepAlive(true)
setSoKeepalive(HttpParams params, enableKeepalive="true") 

How did I resolve my issue?

• Set the HttpConnectionParams.setSoKeepAlive(params, true)
• Catch the SSLException and check for the exception message for Connection reset by peer
• If exception is found, store the download/read progress and create a new connection.
• If possible resume the download/read else restart the download

Answer 2

If using Nginx and getting a similar problem, then this might help:

Scan your domain on this sslTesturl, and see if the connection is allowed for your device version.

If lower version devices(like < Android 4.4.2 etc) are not able to connect due to TLS support, then try adding this to your Nginx config file,

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

Answer 3

we had this same issue starting this morning and goti it solved... hope this helps...

SSL on IIS 8

1. Everything was working fine yesterday and last night our SSL was updated on the IIS site.
2. While checking out the site Bindings to the SSL noticed that IIS8 has a new checkbox Require Server Name Indication, it was not checked so preceded to enable it.
3. That triggered the problem.
4. Went back to IIS, disabled the checkbox.... Problem Solved!!!!

Hope this helps!!!

Answer 4

Android Supports SSL implementation by default except for Android N (API level 24) and below Android 5.1 (API level 22)
I was getting the error when making the API call below API level 22 devices after implementing SSL at the server side; that was while creating OkHttpClient client object, and fixed by adding connectionSpecs() method OkHttpClient.Builder class.

the error received was

response failure: javax.net.ssl.SSLException: SSL handshake aborted: ssl=0xb8882c00: I/O error during system call, Connection reset by peer

so I fixed this by added the check like

if ( Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP_MR1) {
            // Do something for below api level 22
            List<ConnectionSpec> specsList = getSpecsBelowLollipopMR1(okb);
            if (specsList != null) {
                okb.connectionSpecs(specsList);
            }
        }

Also for the Android N (API level 24); I was getting the error while making the HTTP call like

HTTP FAILED: javax.net.ssl.SSLHandshakeException: Handshake failed

and this is fixed by adding the check for Android 7 particularly, like

if (android.os.Build.VERSION.SDK_INT == Build.VERSION_CODES.N){
            // Do something for naugat ; 7
            okb.connectionSpecs(Collections.singletonList(getSpec()));
        }

So my final OkHttpClient object will be like:

         OkHttpClient client
         HttpLoggingInterceptor httpLoggingInterceptor2 = new
         HttpLoggingInterceptor();
         httpLoggingInterceptor2.setLevel(HttpLoggingInterceptor.Level.BODY);

         OkHttpClient.Builder okb = new OkHttpClient.Builder()
                 .addInterceptor(httpLoggingInterceptor2)
               .addInterceptor(new Interceptor() {
                     @Override
                     public Response intercept(Chain chain) throws IOException {
                         Request request = chain.request();
                         Request request2 = request.newBuilder().addHeader(AUTH_KEYWORD, AUTH_TYPE_JW + " " + password).build();
                         return chain.proceed(request2);
                     }
                 }).connectTimeout(30, TimeUnit.SECONDS)
                 .writeTimeout(30, TimeUnit.SECONDS)
                 .readTimeout(30, TimeUnit.SECONDS);

         if (android.os.Build.VERSION.SDK_INT == Build.VERSION_CODES.N){
             // Do something for naugat ; 7
             okb.connectionSpecs(Collections.singletonList(getSpec()));
         }

         if ( Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP_MR1) {
             List<ConnectionSpec> specsList = getSpecsBelowLollipopMR1(okb);
             if (specsList != null) {
                 okb.connectionSpecs(specsList);
             }
         }

         //init client
         client = okb.build();

getSpecsBelowLollipopMR1 function be like,

   private List<ConnectionSpec> getSpecsBelowLollipopMR1(OkHttpClient.Builder okb) {

        try {

            SSLContext sc = SSLContext.getInstance("TLSv1.2");
            sc.init(null, null, null);
            okb.sslSocketFactory(new Tls12SocketFactory(sc.getSocketFactory()));

            ConnectionSpec cs = new ConnectionSpec.Builder(ConnectionSpec.MODERN_TLS)
                    .tlsVersions(TlsVersion.TLS_1_2)
                    .build();

            List<ConnectionSpec> specs = new ArrayList<>();
            specs.add(cs);
            specs.add(ConnectionSpec.COMPATIBLE_TLS);

            return specs;

        } catch (Exception exc) {
            Timber.e("OkHttpTLSCompat Error while setting TLS 1.2"+ exc);

            return null;
        }
    }

The Tls12SocketFactory class will be found in below link (comment by gotev):

https://github.com/square/okhttp/issues/2372

For more support adding some links below this will help you in detail,

https://developer.android.com/training/articles/security-ssl

D/OkHttp: <-- HTTP FAILED: javax.net.ssl.SSLException: SSL handshake aborted: ssl=0x64e3c938: I/O error during system call, Connection reset by peer

Answer 5

Another possible cause for this error message is if the HTTP Method is blocked by the server or load balancer.

It seems to be standard security practice to block unused HTTP Methods. We ran into this because HEAD was being blocked by the load balancer (but, oddly, not all of the load balanced servers, which caused it to fail only some of the time). I was able to test that the request itself worked fine by temporarily changing it to use the GET method.

The error code on iOS was: Error requesting App Code: Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost."

Answer 6

I was experiencing this error on Android 5.1.1 devices sending network requests using okhttp/4.0.0-RC1. Setting header Content-Length: <sizeof response> on the server side resolved the issue.

Answer 7

My problem was with TIMEZONE in emulator genymotion. Change TIMEZONE ANDROID EMULATOR equal TIMEZONE SERVER, solved problem.

reference

Answer 8

Well, gracefully handling exceptions can be very difficult.

ISSUE: I got this javax.net.ssl.SSLException error whenever i make request to my web service knowing fully well that i have exhausted the internet data bundle i subscribed for, but my mobile app confirmed that i am connected to the internet after checking for Connectivity.NetworkAccess == NetworkAccess.Internet. And the worst part of this is that the javax.net.ssl.SSLException error crashes the app at any point the data bundle is exhausted.

MY CASE: In my case, i was testing for what can go wrong if users tries to login when there is internet connection, but there is no data bundle subscription available for the particular ISP they are using. Well, the app crashes with javax.net.ssl.SSLException error, that was the result i got.

MY SOLUTION: I searched Google on the best way to handle this, but the answers i got were specific to different persons so i decided to figure out how to implement mine.

Firstly, as general thing i surrounded my web request code with try-catch block as you can see below;

    public static async Task<LoginResponse> Login(string email, string password)
    {
        try
        {
            var login = new Login()
            {
                email_address = email,
                password = password
            };

            var httpClient = new HttpClient();
            var json = JsonConvert.SerializeObject(login);
            var content = new StringContent(json, Encoding.UTF8, "application/json");
            var response = await httpClient.PostAsync(AppBaseUrl.ApiBaseUrl + "Users/LoginUser", content).ConfigureAwait(false);
            var jsonResult = await response.Content.ReadAsStringAsync();

            var apiResponse = JsonConvert.DeserializeObject<LoginResponse>(jsonResult);
            
            return apiResponse;
        }
        catch (Exception ex)
        {
            return new LoginResponse
            {
                code = 0,
                responsebody = null,
                message = ex.Message
            };
        }
    }

The focus of the above lines code is in the Catch block where i am returning the Login response from my API. Note: code = 0 means the web request was not successful.

And in my ViewModel i added the following lines of code below;

var loginResponse = await Login(Email, Password);

if (loginResponse.code == 1)
{
     // Do something when Login is successful.
}
else
{
     // Do something when Login is not successful.
     await Application.Current.MainPage.DisplayAlert("Oops!", loginResponse.message + ". Try again", "Ok"); // This line will display a message like "Connection closed by peer. Try again"
}

I have attached the screenshot of my app running on a physical Android device. Screenshot of the app

FINAL THOUGHT: I still believe my own error handling method in this case can even be improved on. And what is good about this is that it's handled cross platform. However, the app don't crash anymore and i hope this helps someone out there.