We have a .Net 2.0 web app which uses a third party assembly to make a webservice call from our web page running at the server to the third party site. The call is made over https. This code has been running without an issue for years until about 2 weeks ago. The vendor did update their certs within the last month. However no other client has reported an issue and some of our own machines work without an issue. On some of our servers when the call is made we get The request was aborted: Could not create SSL/TLS secure channel. On other servers there is no issue. The servers are all Windows 2003 running the same code base. We turned on .Net tracing and captured traces on a good server and two bad ones. We've verified the certificate being returned to all the servers is the same. The .Net trace shows one of the InitializeSecurityContext calls returning with a returned code=Illegal Message. I've verified schannel.dll, secur32.dll, and system.net.dll are the same versions on all servers. I've also verified that the CA is Trusted (its Verisign).
At this point I'm looking for any troubleshooting ideas.
Log excerpts below. The first is from a server that gets the error. Note that in the logs for the bad servers the byte receive sequence is always 204, 5, 2, error. On the good server the byte sequence is always 204, 5, 1, continueneeded.
Bad Server
System.Net.Sockets Verbose: 0 : [15784] Exiting Socket#50912888::Send() -> 204#204
System.Net.Sockets Verbose: 0 : [15784] Socket#50912888::Receive()
System.Net.Sockets Verbose: 0 : [15784] Data from Socket#50912888::Receive
System.Net.Sockets Verbose: 0 : [15784] 00000000 : 15 03 00 00 02 : .....
System.Net.Sockets Verbose: 0 : [15784] Exiting Socket#50912888::Receive() -> 5#5
System.Net.Sockets Verbose: 0 : [15784] Socket#50912888::Receive()
System.Net.Sockets Verbose: 0 : [15784] Data from Socket#50912888::Receive
System.Net.Sockets Verbose: 0 : [15784] 00000005 : 02 28 : .(
System.Net.Sockets Verbose: 0 : [15784] Exiting Socket#50912888::Receive() -> 2#2
System.Net Information: 0 : [15784] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 10709bc8:189fc88, targetName = transform.documentmailbox.net, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [15784] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=IllegalMessage).
System.Net.Sockets Verbose: 0 : [15784] Socket#50912888::Dispose()
System.Net Error: 0 : [15784] Exception in the HttpWebRequest#44205226:: - The request was aborted: Could not create SSL/TLS secure channel.
System.Net Error: 0 : [15784] Exception in the HttpWebRequest#44205226::EndGetRequestStream - The request was aborted: Could not create SSL/TLS secure channel.
Good Server:
System.Net.Sockets Verbose: 0 : [0244] Exiting Socket#56654665::Send() -> 204#204
System.Net.Sockets Verbose: 0 : [0244] Socket#56654665::Receive()
System.Net.Sockets Verbose: 0 : [0244] Data from Socket#56654665::Receive
System.Net.Sockets Verbose: 0 : [0244] 00000000 : 14 03 00 00 01 : .....
System.Net.Sockets Verbose: 0 : [0244] Exiting Socket#56654665::Receive() -> 5#5
System.Net.Sockets Verbose: 0 : [0244] Socket#56654665::Receive()
System.Net.Sockets Verbose: 0 : [0244] Data from Socket#56654665::Receive
System.Net.Sockets Verbose: 0 : [0244] 00000005 : 01 : .
System.Net.Sockets Verbose: 0 : [0244] Exiting Socket#56654665::Receive() -> 1#1
System.Net Information: 0 : [0244] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = b2310:174420, targetName = transform.documentmailbox.net, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [0244] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
System.Net.Sockets Verbose: 0 : [0244] Socket#56654665::Receive()
System.Net.Sockets Verbose: 0 : [0244] Data from Socket#56654665::Receive
System.Net.Sockets Verbose: 0 : [0244] 00000000 : 16 03 00 00 38 : ....8
System.Net.Sockets Verbose: 0 : [0244] Exiting Socket#56654665::Receive() -> 5#5
System.Net.Sockets Verbose: 0 : [0244] Socket#56654665::Receive()
System.Net.Sockets Verbose: 0 : [0244] Data from Socket#56654665::Receive
System.Net.Sockets Verbose: 0 : [0244] 00000005 : C0 44 EB FF 6A 88 AD DA-2C 5A 74 99 AD 11 CE 16 : .D..j...,Zt.....
System.Net.Sockets Verbose: 0 : [0244] 00000015 : 4B 10 29 D7 DD 4E A0 83-E9 DE EB BD 37 2F 81 FB : K.)..N......7/..
System.Net.Sockets Verbose: 0 : [0244] 00000025 : D4 9C 99 6C FB A0 CA 6B-1A 4E 7A CA B9 39 1B 91 : ...l...k.Nz..9..
System.Net.Sockets Verbose: 0 : [0244] 00000035 : 7B 26 B1 01 8C FD C1 08- : {&......
System.Net.Sockets Verbose: 0 : [0244] Exiting Socket#56654665::Receive() -> 56#56
System.Net Information: 0 : [0244] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = b2310:174420, targetName = transform.documentmailbox.net, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
System.Net Information: 0 : [0244] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=OK).
