SSL_write reports success but fails in Safari

Question

I have my own implementation of a webserver that utilizes openssl.

I am using the blocking version of SSL_write to transmit data and the latest Safari on OSX.

My logic is as follows:

num_bytes = 0;
while( num_bytes < bytes_to_send) {
  int n = SSL_write(...);
  if (n <= 0) {
     ... // Handle error, break loop
  }
  num_bytes += n;
}
// Close the socket

There is an infrequent problem I am encountering where Safari reports Failed to load resource: The network connection was lost.

When I check the resource in the developer pane under Safari, it shows that there were less bytes received than expected.

I have debugged my code and determined SSL_write is reporting that all bytes were transferred before I close the connection.

When I do not close the connection, this bug no longer occurs.

This leads me to the following conclusion:

SSL_write incorrectly reports a number of bytes were successfully transferred before it has actually occurred, and that closing the connection immediately after cuts the transfer short.

Am I correct, or is there a detail about SSL_write that I am missing?

Update:

After examining the way I process SSL_shutdown, I am seeing a couple of SSL_ERROR_SYSCALL errors being reported. This may or may not have anything to do with my problem since these errors are not directly returned after the affected http transfers. Strangely, during the affected http transfers no errors are reported from SSL_shutdown other than the retry error for blocking calls.

Shutdown Code

// First shutdown SSL
SSL_CTX_free(ctx);
if (SSL_shutdown(ssl) == 0) {
  // Call SSL Shutdown again
  SSL_shutdown(ssl);
}
SSL_free(ssl);

// Now shutdown port
struct linger linger;
linger.l_onoff = 1;
linger.l_linger = 10;
setsockopt(socket, SOL_SOCKET, SO_LINGER, (char *)&linger,
           sizeof(linger));

shutdown(socket, SHUT_WR);
while(recv(...) > 0) { }
closesocket(socket);

Update #2

I set up wireshark with ssl decryption to capture the error. Here is what it looks like:

Notice the first green line, which shows the HTTP request from Safari for PatientSearch.js . Now the second green line shows the HTTP response from my webserver. This response contains the file in its entirety, followed by the disconnection protocol.

Update #3

This is getting more mysterious by the second. After examining a wireshark trace for a successful transfer, and there is no difference in the protocol pattern in the successful transfer vs the broken transfer. Completely confused now.

Answer 1

Your send loop is not keeping track of the number of sent bytes correctly. It should look more like this instead:

unsigned char *data = ...;
while (bytes_to_send > 0) {
  int num_sent = SSL_write(ssl, data, bytes_to_send);
  if (num_sent <= 0) {
    // handle error as needed...
    break;
  }
  data += num_sent;
  bytes_to_send -= num_sent;
}
// Close the socket

Or this:

unsigned char *data = ...;
int num_total_sent = 0;
while (num_total_sent < bytes_to_send) {
  int num_sent = SSL_write(ssl, &data[num_total_sent], bytes_to_send - num_total_sent);
  if (num_sent <= 0) {
    // handle error as needed...
    break;
  }
  num_total_sent += num_sent;
}
// Close the socket

---

Update:

SSL_write incorrectly reports a number of bytes were successfully transferred before it has actually occurred

The return value reports the number of bytes it accepted from you. That does not guarantee that those bytes were actually transmitted yet. They are encrypted and used to create an outgoing SSL frame, which is sitting in the underlying socket's outbound buffer waiting for the kernel to actually transmit it in the background. That transmission may take some time, especially if the Nagle algorithm is enabled on the socket (which it usually is by default).

closing the connection immediately after cuts the transfer short.

That is possible, depending on how you are closing the connection, how the socket's linger option is configured, etc. By default, closing a socket gracefully should allow the kernel to take a few extra moments in the background to write out any pending data before it then actually closes the connection. You usually have to take extra measures to change that behavior.