WCF Service calls always fail after 30 seconds with (502) Bad Gateway

Question

We have a WCF service (BasicHttpBinding) which will always fail after 30 seconds. Calls under 30 seconds complete without error. Anything over 30 seconds will fail with a 502 Bad Gateway exception:

System.Net.WebException: The remote server returned an error: (502) Bad Gateway.

But yet the WCF call continues to run in the background (and will eventually complete). We have confirmed that the BasicHttpBinding - Binding - sendTimeout (in web.config) is greater than 30 seconds (actually set to 5 minutes). We confirmed this both on the client and the server.

Here is the full stack trace:

System.ServiceModel.ProtocolException: The remote server returned an unexpected response: (502) Bad Gateway. ---> System.Net.WebException: The remote server returned an error: (502) Bad Gateway.
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   --- End of inner exception stack trace ---

Server stack trace: 
   at System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory factory, WebException responseException, ChannelBinding channelBinding)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

Any ideas where this 30 second "timeout" is coming from or why a 502 Bad Gateway error is returned?

SOLUTION: We are using the IIS7 Application Request Routing Module which has Proxy settings of its own. The Proxy settings have a default timeout of 30 seconds. Increasing this to 600 seconds (10 minutes) solved our problem. The Bad Gateway error is not completely correct but WCF Trace Viewer (see answer) helped see that the problem was not the service itself but an issue in between the client and the wcf service.

score 5 · Accepted Answer · edited May 23 '17 at 12:30

5

You may want to look try modifying the other timeout config values:

closeTimeout, openTimeout, recieveTimeout.

See this MSDN post for information on the config items, summary below:

Client side:

SendTimeout is used to initialize the OperationTimeout, which governs the whole interaction for sending a message (including receiving a reply message in a request-reply case). This timeout also applies when sending reply messages from a CallbackContract method.

OpenTimeout and CloseTimeout are used when opening and closing channels (when no explicit timeout value is passed).

Server side:

Send, Open, and Close Timeout same as on client (for Callbacks).

ReceiveTimeout is used by ServiceFramework layer to initialize the session-idle timeout.

ADDED:

The only other thing I can suggest is to use the WCF Service Trace Viewer to get to the bottom of what's causing the issue. See this SO Post if you require details on how to use it.

edited May 23 '17 at 12:30

Community

1
1

answered May 04 '10 at 14:37

Tanner

22,205
9
65
83

We have tried increasing closeTimeout, openTimeout, and receiveTimeout to 5 minutes (on client and server) and still see the issue. – Jeff Widmer May 04 '10 at 15:20
Thanks for the info on the WCF Service Trace Viewer. I used it to gather a trace on both the client and server but all I am seeing from it is that the server is completing properly but the client ends up with a "Received bad HTTP Response" with the internal message being "The remote server returned an unexpected response: (502) Bad Gateway." The server completes after 42 seconds and the client gets the error at 31 seconds. It is always 31 seconds for the client. – Jeff Widmer May 05 '10 at 18:42
Figured it out. We are using the IIS7 Application Request Routing Module which has Proxy settings one of which was a 30 second timeout. Using WCF Service Trace Viewer did help me to understand that the WCF service was not the problem but something in between the client and the service. – Jeff Widmer May 05 '10 at 20:07

score 1 · Answer 2 · answered Sep 06 '12 at 22:05

1

IIS -> Advanced Settings -> Connection Limits

Increase that number (in seconds) to the desired amount.

Hope this helps any Googlers out there!

answered Sep 06 '12 at 22:05

Dylan Douglas

128
5

score 0 · Answer 3 · edited Oct 06 '11 at 23:18

I've hit this very problem myself today. Uploading a file to a WCF webservice from a SL4 app kept raising a ConnectionTimeout exception after 30secs.

I found the cause of the problem to be the use of the WebRequest.RegisterPrefix method as recommended by Microsoft to overcome Fault Exception handling in Silverlight:

bool registerResult = WebRequest.RegisterPrefix("http://", WebRequestCreator.ClientHttp);

See http://msdn.microsoft.com/en-us/library/ee844556(v=vs.95).aspx

WCF Service calls always fail after 30 seconds with (502) Bad Gateway

3 Answers3