It's true that there is no explicit reference to the impact of TLS 1.3 on SSLEngine in its Javadoc in JDK 11, and there were no changes in its methods.
However the fifth item (closure) in the list of phases of SSLEngine was updated in the general description at the start of its Javadoc in JDK 11:
Closure - When the connection is no longer needed, the client and the
server applications should each close both sides of their respective
connections. For SSLEngine objects, an application should call
closeOutbound() and send any remaining messages to the peer. Likewise,
an application should receive any remaining messages from the peer
before calling closeInbound(). The underlying transport mechanism can
then be closed after both sides of the SSLEngine have been closed. If
the connection is not closed in an orderly manner (for example
closeInbound() is called before the peer's write closure notification
has been received), exceptions will be raised to indicate that an
error has occurred. Once an engine is closed, it is not reusable: a
new SSLEngine must be created.
The change is also discussed in Oracle's Release Notes for JDK 11:
TLS 1.3 Half-Close Policy
A new system property,
jdk.tls.acknowledgeCloseNotify, has been added. The default value of
the system property is false. If the system property is set to true, a
corresponding close_notify alert will be sent when receiving a
close_notify alert, and the connection will be duplex closed.
TLS 1.2 and prior versions use a duplex-close policy, while TLS 1.3
uses a half-close policy. The inbound and the outbound close_notify
alerts for TLS 1.3 are independent. When upgrading to TLS 1.3,
unexpected behavior can occur if your application shuts down the
(D)TLS connection by using only one of the SSLEngine.closeInbound() or
SSLEngine.closeOutbound() APIs, but not both in each side of the
connection. If your application exhibits unexpected hangs or timeouts
when the underlying (D)TLS transportation is not duplex closed, you
may need to set this property to true.
Note that when a TLS/DTLS connection is no longer needed, the client
and server applications should each close both sides of their
respective connection.
So setting jdk.tls.acknowledgeCloseNotify to true might resolve your specific concern about timeouts when using SSlEngine with TLS 1.3:
If your application exhibits unexpected hangs or timeouts when the underlying (D)TLS transportation is not duplex closed, you may
need to set this property to true.
The Release Notes document also links to the closed JDK bug JDK-8208526 : TLS 1.3 half-close and synchronization issues which discusses the change in even greater detail.
The related (and closed) bug JDK-8207009 : TLS 1.3 half-close and synchronization issues may also be of interest.
Other References:
- See "Appendix D. Backward Compatibility" of RFC 8446: "The Transport Layer Security (TLS) Protocol Version 1.3" (pp. 138-141).
- There's a brief discussion of compatibility between TLS 1.3 and earlier releases in this Oracle video "Monday Technical Sessions: Moscone West 2004" between 2:53:37 and 2:56:35.
