Writing bytes directly to OpenSSL Connection, without TCP

Question

I'm implementing a RADIUS server with support for EAP(PEAP) authentication. The RADIUS relies on UDP and each packet will contain an EAP message with the authentication data. To simplify, each EAP (PEAP) message contains TLS packets so the client first will send an EAP packet containing TLS Client Handshake, which the server will process, and then return the TLS Server Handshake, and so on with the following packages.

I've been able to implement a simple TLS server over TCP, but what I need is write TLS bytes directly in the SSL connection.

SSLConn_init_thread();
SSL_load_error_strings();
ERR_load_crypto_strings();
SSL_library_init();

SSL_CTX *ctx;
...
ctx = SSL_CTX_new(SSLv23_server_method());
SSL_CTX_use_PrivateKey(...);
SSL_CTX_use_certificate(...);

ssl = SSL_new(ctx); // I want to read/write TLS packets directly with this object

I've looked into OpenSSL docs but it's not very well documented. In this guide says one can make the handshake "by hand" using the SSL_set_accept_state() with SSL_read and SSL_write, but I don't understand exactly how to do that.

Any advice with this or about handling OpenSSL connections without relying on TCP would be very appreciated.

You're looking for DTLS or [Datagram Transport Layer Security](https://en.wikipedia.org/wiki/Datagram_Transport_Layer_Security). See [DTLS examples?](http://openssl.6102.n7.nabble.com/DTLS-examples-td2143.html) from the OpenSSL User List. — jww, Mar 27 '14 at 19:48
@noloader Thank you, but TLS data does not go directly in UDP datagrams, but in RADIUS EAP messages (which are transported via UDP). — S. A., Mar 27 '14 at 23:24
@SteffenUllrich thank you, I'm looking into this, it seems to be the way to go. — S. A., Mar 27 '14 at 23:25
Something is wrong with this question. There are no such things as "TLS packets". TLS is layered over TCP, which is a byte-stream protocol. The mapping of TLS protocol data units to packets is totally arbitrary. — David Schwartz, May 02 '16 at 19:59
@David Schwartz and the RFC refers to them as Records not PDUs, no need to be pedantic. — Arran Cudbard-Bell, Apr 22 '17 at 23:46

score 5 · Accepted Answer · edited Jun 20 '20 at 09:12

I ended up reading the book Network Security with OpenSSL and found Memory BIOs were what i needed.

Network Security With OpenSSL

A memory BIO treats a memory segment the same as a file or socket, and can be created by using BIO_s_mem to obtain a BIO_METHOD object suitable for use with BIO_new and BIO_set

BIO_s_mem()

BIO_s_mem() return the memory BIO method function.

A memory BIO is a source/sink BIO which uses memory for its I/O. Data written to a memory BIO is stored in a BUF_MEM structure which is extended as appropriate to accommodate the stored data.

Any data written to a memory BIO can be recalled by reading from it. Unless the memory BIO is read only any data read from it is deleted from the BIO.

Basically you can read/write encrypted data directly to the SSL connection using the memory BIOs:

// setup SSL_context...
ssl = SSL_new(ctx);

// Create read/write BIOs 
rbio = BIO_new(BIO_s_mem());
wbio = BIO_new(BIO_s_mem());

SSL_set_bio(ssl, rbio, wbio);

if (/* Server */) {
  SSL_set_accept_state(ssl); 
} else {
  SSL_set_connect_state(ssl);
}

To write/read from BIOs:

BIO_read(wbio, buffer, strlen(buffer));
BIO_write(wbio, buffer, strlen(buffer));

given the circumstances, are you sure this solution is safe? — Leeor, Apr 18 '14 at 16:09
Well, it's safe as far as you have a non vulnerable OpenSSL version (with "the circumstances" you mean [HeartBleed](http://heartbleed.com/) right?). — S. A., Apr 19 '14 at 03:44
How do you propose `BIO_read(wbio, buffer, strlen(buffer));` correlates the size of the payload in wbio given `strlen(buffer)`? You don't really understand what you're taking about. — , Oct 25 '21 at 14:42

score 5 · Answer 2 · answered Mar 24 '17 at 06:44

In ascii diagram terms, here's what the flow of bytes loops like. SSL_read/SSL_write are used to pass unencrypted bytes in-to/out-of the SSL object, and BIO_read/BIO_write are used to get corresponding encrypted bytes in-to/out-of the SSL object. It's the up to you to then transfer the encrypted bytes over a network connection.

Git hub example of non blocking sockets / SSL

(https://gist.github.com/darrenjs/4645f115d10aa4b5cebf57483ec82eca)

  +------+                                    +-----+
  |......|--> read(fd) --> BIO_write(rbio) -->|.....|--> SSL_read(ssl)  --> IN
  |......|                                    |.....|
  |.sock.|                                    |.SSL.|
  |......|                                    |.....|
  |......|<-- write(fd) <-- BIO_read(wbio) <--|.....|<-- SSL_write(ssl) <-- OUT
  +------+                                    +-----+
          |                                  |       |                     |
          |<-------------------------------->|       |<------------------->|
          |         encrypted bytes          |       |  unencrypted bytes  |

Writing bytes directly to OpenSSL Connection, without TCP

2 Answers2