1

I use Tokio to create plain TCP sockets, call tokio::io::split() and the read/write halves get handed to separate threads. They use the async socket read/write APIs with await to accomplish the IO. Our data flow is fairly isolated in the in/out directions, so this model works well in our case. So far so good.

Now am looking at adding TLS support on top. Some of the TLS libraries don't allow splitting the stream for various reasons:

  • tokio-rustls (implemented with rustls) allows splitting, but it is relatively new

  • I would prefer to use tokio-openssl (implemented with openssl), which has been around for much longer, but openssl does not support it. This is probably because events like TLS renegotiation need to be propagated to the read/write halves (rustls manages it because it a native Rust implementation).

So the same thread needs to do the reads/writes. This implies that the socket needs to become non-blocking now: can't wait for data to come in, as data may need to be sent out immediately (and vice versa).

If I understand correctly, the Tokio/await paradigm doesn't make sense with the non-blocking sockets. Is my understanding correct?

Any other ideas in this scenario also welcome. I hope we don't need to abandon Tokio after all the effort put in so far.

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
rusty
  • 929
  • 1
  • 5
  • 10
  • 1
    *the socket needs to become non-blocking now* — it probably already **is** non-blocking. *the Tokio/await paradigm doesn't make sense with the non-blocking sockets* — that's... almost the entire reason that the entire `async` ecosystem **exists**: to make dealing with non-blocking IO easier to reason about. Can you share why you believe the polar opposite? – Shepmaster Apr 07 '20 at 20:21
  • 1
    The non-blocking here refers to the socket state: O_NONBLOCK flag.The usual method is: the socket is in blocking mode, and a `read().await` waits for data to come in. The `async` system makes sure thread is relinquished while waiting. But most of the r/w methods take `&mut self`, so while this await is pending, can't do a write. Which is why I was thinking the socket itself needs to be in non-blocking mode. This allows the socket to be "polled" if data is present without actually (async) blocking. Hope I answered your question – rusty Apr 07 '20 at 20:35
  • *refers to the socket state: `O_NONBLOCK` flag* — sure, and [`tokio::net::TcpStream`](https://docs.rs/tokio/0.2.16/tokio/net/struct.TcpStream.html) calls [`mio::net::TcpStream`](https://docs.rs/mio/0.7.0/mio/net/struct.TcpStream.html) which says: "A non-blocking TCP stream" and it [sets that flag](https://github.com/tokio-rs/mio/blob/v0.7.0/src/sys/unix/net.rs#L22-L57). – Shepmaster Apr 07 '20 at 20:43
  • *Some of the TLS libraries don't allow splitting the stream for various reasons* — you'll need to [edit] your question to be more specific; ideally providing a [MRE]. [`tokio::io::split`](https://docs.rs/tokio/0.2.16/tokio/io/fn.split.html) only requires that the underlying type implement `AsyncRead` + `AsyncWrite`. The first search result I found for "tokio tls" implements both of those traits. – Shepmaster Apr 07 '20 at 20:54
  • *rustls allows splitting* — then you've answered your own question: **no**, you don't need to move away from Tokio due to the inability to split TLS streams. – Shepmaster Apr 07 '20 at 21:03
  • :-) But it is relatively new, would like to use openssl preferably, which has been around for much longer. – rusty Apr 07 '20 at 21:08
  • To be clear, talking about tokio-openssl and tokio-rustls here – rusty Apr 07 '20 at 21:11
  • 1
    Regarding the linked thread, it explicitly says you _can_ split by putting it in a mutex. This is essentially what `tokio::io::split` does. – Alice Ryhl Apr 07 '20 at 21:17
  • Thanks, looks like I was splitting it wrong (splitting the TcpStream instead of the enclosing SslStream<>) – rusty Apr 07 '20 at 23:05

1 Answers1

4

It's true that async/await enabled TLS libraries such as tokio-tls require the provided stream to not have been split, however once you have wrapped your stream in a TLS layer, you can split that wrapped stream using tokio::io::split.

Using streams in this way correctly handles all details regarding blocking and non-blocking IO. You do not need to manually configure flags such as O_NONBLOCK, since Tokio's TcpStream and tokio-tls's TlsStream handle these details for you behind the scenes.

Using a library that provides blocking sockets would naturally not be compatible with Tokio. This is not new, and is for the same reasons that you can't use std::net::TcpStream within Tokio, as it is a blocking stream. Tokio provides alternate stream types for these purposes to avoid these issues.

If you wanted to use a non-async/await enabled ssl crate, you can perform the crypto on in-memory buffers, and manually write the encrypted data using Tokio's TcpStream. The async/await enabled ssl libraries all function in this way.

Alice Ryhl
  • 3,574
  • 1
  • 18
  • 37