3

I am attempting to send a self-signed client certificate using HttpClient with the following code:

var handler = new HttpClientHandler();
handler.ClientCertificateOptions = ClientCertificateOption.Manual; 
handler.ClientCertificates.Add(GetClientCertificate()); //Loads cert from a .pfx file
var client = new HttpClient(handler);
PerformRequest(client);

I've seen a dozen SO posts related to this, but none have resolved my issue.

Things to note:

  • I have verified via Wireshark that the server is requesting a client cert, but HttpClient is not sending one.

When sent via HttpClient

  • The exact same certificate (same .pfx file is used) is sent when I use Postman to perform the request

When sent via Postman

  • I have tried forcing the TLS version via handler.SslProtocols to 1.0, 1.1, 1.2 -- none work
  • The X509Certificate2 returned from GetClientCertificate() has a private key
  • This is on .NET Framework 4.7.2 -- I can't change this

I have been troubleshooting this for 3 days now and even read through the reference source trying to figure out why my cert isn't included, but I can't find a reason.
I suspect somewhere deep inside the HttpClient implementation, my cert is getting rejected because it is self-signed.

How can I force it to send my cert? (the server maintains a whitelist, so I don't care if it thinks my cert is invalid). Or, at bare minimum, how can I get any sort of debugging information out of this? Is there any way to get a reason why a cert is rejected?

Update:

After enabling tracing, I have found that .NET is correctly selecting my certificate:

System.Net Information: 0 : [23960] SecureChannel#64538993 - Selected certificate: <omitted>
System.Net Information: 0 : [23960] SecureChannel#64538993 - Left with 1 client certificates to choose from.
System.Net Information: 0 : [23960] SecureChannel#64538993 - Trying to find a matching certificate in the certificate store.
System.Net Information: 0 : [23960] SecureChannel#64538993 - Locating the private key for the certificate: <omitted>
System.Net Information: 0 : [23960] SecureChannel#64538993 - Certificate is of type X509Certificate2 and contains the private key.
System.Net Information: 0 : [23960] SecureChannel#64538993::.AcquireClientCredentials, new SecureCredential() (flags=(ValidateManual, NoDefaultCred, SendAuxRecord), m_ProtocolFlags=(Tls12Client), m_EncryptionPolicy=RequireEncryption)
System.Net Information: 0 : [23960] AcquireCredentialsHandle(package = Microsoft Unified Security Protocol Provider, intent  = Outbound, scc     = System.Net.SecureCredential)
System.Net Information: 0 : [23960] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 21c5cd98:21cd2108, targetName = <omitted>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [23960] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=100, returned code=ContinueNeeded).
... etc

However, it is still not being sent. Any more ideas on where to look would be greatly appreciated.

MrZander
  • 3,031
  • 1
  • 26
  • 50
  • Do you see a request in both postman and c#? If a request is sent than TLS is working. A certificate is not sent with TLS. The server sends a certificate block with name of certificate. See Wiki at bottom of page for the TLS protocol : https://en.wikipedia.org/wiki/Transport_Layer_Security – jdweng May 26 '21 at 23:59
  • 1
    You may be able to get more diagnostic information by enabling a .NET trace on the `System.Net` name, which includes "SSL debug information (invalid certificates, missing issuers list, and client certificate errors)." See [link](https://learn.microsoft.com/en-us/dotnet/framework/network-programming/how-to-configure-network-tracing). – John Wu May 27 '21 at 00:21
  • @jdweng I'm not sure I understand what you're asking. The server is sending "Certificate Request" during the "Server Hello". I do see a request from both, but the difference is postman responds with the client certificate and C# does not. – MrZander May 27 '21 at 15:29
  • @JohnWu Thank you! That is a ton of help. However, now I am even more confused. The trace shows that it is accepting my cert, so why is it not getting sent down the wire? – MrZander May 27 '21 at 15:42
  • You have a connection and are sending the certificate in the body of the connection after it completes. The certificate is not being used as part of the TLS authentication. TLS occurs first. When TLS fails then there is no request. You get a request so the TLS was successful. – jdweng May 27 '21 at 15:43
  • @MrZander that's not an HttpClient issue. By definition a self-signed certificate is invalid. Its authenticity simply can't be verified by any Certificate Authority. The good solution is to add that certificate to your dev machine's trusted certificates – Panagiotis Kanavos May 27 '21 at 15:51
  • @jdweng The server (IIS) is configured to accept client certificates, but not validate them. That is left up to the application server. So yes, TLS is successful in both cases, but the server is only able to authenticate the Postman request because C# does not send a cert during the "Client-authenticated TLS handshake" part of the TLS protocol. See the 7th bullet point on that section of the wiki "The client responds with a Certificate message, which contains the client's certificate." -- C# isn't doing this. – MrZander May 27 '21 at 15:54
  • How you add the certificate to the trusted list depends on the OS. With Powershell, you can use `Import-Certificate -FilePath $certFilePath -CertStoreLocation 'Cert:\LocalMachine\Root'` – Panagiotis Kanavos May 27 '21 at 15:55
  • Check the [Import-Certificate](https://learn.microsoft.com/en-us/powershell/module/pki/import-certificate?view=windowsserver2019-ps) docs. You may be able to use a different store, eg the current user's instead of the machine's – Panagiotis Kanavos May 27 '21 at 15:57
  • 1
    @PanagiotisKanavos : Why does Postman work??? – jdweng May 27 '21 at 16:09
  • @jdweng [does it? Or did the OP unblock self-signed certificates](https://blog.postman.com/self-signed-ssl-certificate-troubleshooting/) ? – Panagiotis Kanavos May 27 '21 at 17:21
  • @jdweng looks like [POSTMAN explicitly disabled the warning in 2020](https://blog.postman.com/2019-in-review-a-great-year-for-postman-product-improvements/) to make testing easier. That doesn't mean *production* code should disable it – Panagiotis Kanavos May 27 '21 at 17:24
  • @PanagiotisKanavos : We do not know. – jdweng May 27 '21 at 17:24
  • @jdweng we do know, because the changes were documented in POSTMAN's blog. [In 2014](https://blog.postman.com/using-self-signed-certificates-with-postman/) when was still a Chrome extension, it used the browser settings and one had to explicitly trust the certificate. In 2019 it still blocked it even as a standalone application. And in 2020 it explicitly disabled the validation – Panagiotis Kanavos May 27 '21 at 17:26
  • @PanagiotisKanavos You are confusing SERVER certificates with CLIENT certificates. My server certificate is signed by a CA, my client cert is self-signed. Regardless, it doesn't matter, I figured it out -- issue was due to the signature algorithm. – MrZander May 27 '21 at 17:29

1 Answers1

1

This answer led me to the solution.

X509Certificate2.PrivateKey was throwing a NotSupportedException because I was using ECD as the signature algorithm. I switched to RSA and now it properly sends the certificate.

What's strange is that tracing showed no issue with the ECD cert, and it was able to successfully identify the private key. I have no idea why this is the case -- it sounds like a bug to me. Nevertheless, RSA will work for my use case and I am tired of debugging this.

MrZander
  • 3,031
  • 1
  • 26
  • 50
  • It's not a bug, it's documented in the [prorperty docs](https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.x509certificates.x509certificate2.privatekey?view=net-5.0) `An AsymmetricAlgorithm object, which is either an RSA or DSA cryptographic service provider.` .NET *does* have ECDsa classes though. – Panagiotis Kanavos May 27 '21 at 17:35
  • You may be able to use [ECDsaCertificateExtensions.CopyWithPrivateKey(X509Certificate2, ECDsa)](https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.x509certificates.ecdsacertificateextensions.copywithprivatekey?view=net-5.0#System_Security_Cryptography_X509Certificates_ECDsaCertificateExtensions_CopyWithPrivateKey_System_Security_Cryptography_X509Certificates_X509Certificate2_System_Security_Cryptography_ECDsa_) to combine certificate and Private key. This method was added in 4.7.2 – Panagiotis Kanavos May 27 '21 at 17:36
  • I understand that `PrivateKey` states that it doesn't support ECDsa, that was just the clue that led me to changing the signature algorithm. But I see no reason why an X509Cert using ECDsa should silently not get sent with my request. The fact that the same cert works with Postman leads me to believe that it is indeed a bug. – MrZander May 27 '21 at 17:39
  • It's documented that this property doesn't accept ECDsa. That's why it throws `NotSupportedException` instead of some other random error - it's explicitly written to reject ECDsa, [even in .NET Core](https://github.com/dotnet/runtime/blob/01b7e73cd378145264a7cb7a09365b41ed42b240/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/X509Certificate2.cs#L253). Have you tried using `CopyWithPrivateKey`? – Panagiotis Kanavos May 27 '21 at 17:48
  • That's all well and good, but what isn't documented is that an ECDsa signed certificate will silently fail to be sent by HttpClient. I'm not working on it any further, I'm just going to use RSA. – MrZander May 27 '21 at 17:56