6

I am trying to create a DTLS "connection" using Schannel under Windows (I am testing under recent Windows 10 version, so all DTLS versions supported by Schannel should be available)

I tried starting from working code to establish a regular TLS connection by following the documentation:

  1. InitializeSecurityContext with null input on the first pass, SECBUFFER_TOKEN & SECBUFFER_ALERT on output
  2. AcceptSecurityContext with SECBUFFER_TOKEN & SECBUFFER_EMPTY on input, SECBUFFER_TOKEN & SECBUFFER_ALERT on output.
  3. Repeat the two steps until they succeed, and then move on to using Encrypt/DecryptMessage

This works perfectly fine in stream mode (ISC_REQ_SEQUENCE_DETECT | ISC_REQ_REPLAY_DETECT | ISC_REQ_CONFIDENTIALITY | ISC_RET_EXTENDED_ERROR | ISC_REQ_ALLOCATE_MEMORY | ISC_REQ_STREAM)

If I try to substitute STREAM with ISC/ASC_REQ_DATAGRAM, my InitializeSecurityContext succeeds with SEC_I_CONTINUE_NEEDED as expected, but my very first AcceptSecurityContext then fails with SEC_E_INVALID_PARAMETER.

I have tried setting grbitEnabledProtocols of my SCHANNEL_CRED to 0 to use the defaults as documented on both sides, I also tried setting it to SP_PROT_DTLS1_X, and I still get the Invalid Parameter return from my first ASC. I have also tried the DTLS_1_0 constants just in case.

Also, all security protocols are enabled by default in my registry settings.

From my understanding of the DTLS RFC, my code is failing at the HelloVerifyRequest step, and, again from my understanding of the RFC, this part requires that the security provider generates a cookie from a few parts of the ClientHello message as well as the source's IP address. However, I could not find any documented way to pass this information to the ASC function.

(I think? :) ) I have searched the entire internet for any working example usage of DTLS under Schannel without any luck. On stackoverflow, I found this question that simply mentions that it is supported: Is DTLS supported by Schannel on Windows 7?, and the linked MSDN article is just a high level overview.

I searched for any usage of the constants that are related to this feature... I searched for any usage of the constants that are related to this (ISC_REQ_DATAGRAM, SP_PROT_DTLS*, SECBUFFER_DTLS_MTU, ...) and the only thing I could find on all search engines I could think of were either copies of sspi.h or sites that index the constants in the Windows API...

I know DTLS is well supported by other implementations (OpenSSL etc), but I would really prefer to stay with Schannel, as other parts of my code currently work just fine with Schannel in TLS mode.

fbrosseau
  • 374
  • 3
  • 9

2 Answers2

5

From Microsoft: There is no documentation for implementing DTLS using Schannel. Known and persistent doc gap.

There are a few differences, but a TLS client or server can be adapted to DTLS fairly easily (a number of customers have done this successfully).

  1. Set SCHANNEL_CRED.grbitEnabledProtocols to SP_PROT_DTLS1_X.
  2. When calling AcceptSecurityContext, pass the client’s SOCKADDR via SECBUFFER_EXTRA.
  3. MTU can be set via SetContextAttributes using constant SECPKG_ATTR_DTLS_MTU where the buffer is just an pointer to a ULONG. [Default is 1096 bytes.]
  4. When ISC/ASC return SEC_I_MESSAGE_FRAGMENT, send this fragment and call ISC/ASC again, in a loop, to get the next fragment (without trying to read data from the network).
  5. Implement timeout and retransmit logic in your application (since schannel does not own the socket).
  6. When receiving fragments, schannel will attempt to eliminate duplicates, re-order and re-assemble, if possible.
  7. SCHANNEL_SHUTDOWN does not apply to DTLS.
Matthew J. Bobowski
  • 76
  • 1
  • 6
  • Considering opening a new question, but maybe you can answer here? After I receive SEC_I_MESSAGE_FRAGMENT from ASC, and call it again to retrieve the next fragment, I am getting SEC_E_INCOMPLETE_MESSAGE and nothing in the output token buffer. (This is when passing a zero-length input token buffer - if I re-pass the original input, I get a repeat of the first fragment as output.) Is there something specific I need to do to indicate to ASC that it should give me the next fragment? – Haddon CD. May 07 '21 at 17:29
  • I take it you are still in your negotiation loop? If so, exit the loop and process the error for encrypted data underflow, queue and process on next recv. To do this check for SECBUFFER_EXTRA from index 1 (second element) returned from ASC. Backup data to remove already processed information from the input buffer. When receiving data again add new data to the backed up data by resizing the stored data buffer and either continue negotiation or call DecryptMessage(). – Matthew J. Bobowski May 21 '21 at 07:27
  • Matthew, thanks (and apologies if I'm mistaken), but that sounds like how to handle SEC_E_INCOMPLETE_MESSAGE. I'm having trouble handling SEC_I_MESSAGE_FRAGMENT, i.e. step 4 of your answer above. I have in fact asked this as a new question (with more detail and sample code that reproduces the issue), which remains unanswered here, if you want to take a look: https://stackoverflow.com/q/67463514/1831696 – Haddon CD. Jul 13 '21 at 21:42
  • My comment was for SEC_I_MESSAGE_FRAGMENT. The difference is you continue in the loop, vs. exit the loop for SEC_E_INCOMPLETE_MESSAGE (data underflow). – Matthew J. Bobowski Jul 15 '21 at 01:35
  • Hi Matthew. For more detail see my question, but when I receive SEC_I_MESSAGE_FRAGMENT, I am continuing in my loop (i.e. sending the output SECBUFFER_TOKEN, then calling ASC again without polling the socket) but I'm not sure what to pass ASC as input on the next iteration. I assume (the same as with SEC_E_INCOMPLETE_MESSAGE) I shouldn't pass repeated SECBUFFER_TOKEN data unless ASC indicates it as unprocessed with an output SECBUFFER_EXTRA, but if I pass a zero length input (either as SECBUFFER_TOKEN or _EMPTY), I just get SEC_E_INCOMPLETE_MESSAGE, instead of the next fragment. Thanks again! – Haddon CD. Jul 27 '21 at 13:09
  • 1
    Thanks to another user, I was able to resolve the issue with my sample code, which turned out to be unrelated to my handling of SEC_I_MESSAGE_FRAGMENT. The fixed code is available here if anyone wants to see an example of what Matthew suggests in his answer: https://gist.github.com/haddoncd/381c5e9542e977ca238ff16229bd9a0e – Haddon CD. Sep 11 '21 at 10:58
-2

You can use https://github.com/mobius-software-ltd/iotbroker.cloud-windows-client As a sample to implement DTLS on windows It does not uses SChannel but netty library. MQTT-SN And CoAP are both supporting DTLS under this project. BR Yulian Oifa

Yulian Oifa
  • 111
  • 7
  • 1
    This question was specifically about SChannel. Outside of SChannel, DTLS can trivially be implemented using OpenSSL and others. Most of your stackoverflow answers appear to be advertisements for your projects. – fbrosseau Feb 06 '21 at 19:23