WCF SOAP Error for Larger Requests

Question

I need to download a few hundred thousand entities from a 3rd party hosted web service. The web service allows me to pull down up to 2,000 entities per web service call, but in practice I have only been able to pull down 50-75 entities at a time because of intermittent errors I receive when I call more records. I'm trying to determine if the problem is due to a setting I need to adjust on my side or if the problem is with the 3rd party web service provider. Here's the error I receive:

An error occurred while receiving the HTTP response to https://some.vendor.com/API/SomeService.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.

The following is a copy of my application configuration settings for binding the web service:

<system.serviceModel>
    <bindings>
        <customBinding>
            <binding name="SomeService">
                <security defaultAlgorithmSuite="Default" authenticationMode="UserNameOverTransport"
                    requireDerivedKeys="true" securityHeaderLayout="Strict" includeTimestamp="true"
                    keyEntropyMode="CombinedEntropy" messageSecurityVersion="WSSecurity11WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10">
                    <localClientSettings cacheCookies="true" detectReplays="false"
                        replayCacheSize="900000" maxClockSkew="00:05:00" maxCookieCachingTime="Infinite"
                        replayWindow="00:05:00" sessionKeyRenewalInterval="10:00:00"
                        sessionKeyRolloverInterval="00:05:00" reconnectTransportOnFailure="true"
                        timestampValidityDuration="00:05:00" cookieRenewalThresholdPercentage="60" />
                    <localServiceSettings detectReplays="false" issuedCookieLifetime="10:00:00"
                        maxStatefulNegotiations="128" replayCacheSize="900000" maxClockSkew="00:05:00"
                        negotiationTimeout="00:01:00" replayWindow="00:05:00" inactivityTimeout="00:02:00"
                        sessionKeyRenewalInterval="15:00:00" sessionKeyRolloverInterval="00:05:00"
                        reconnectTransportOnFailure="true" maxPendingSessions="128"
                        maxCachedCookies="1000" timestampValidityDuration="00:05:00" />
                    <secureConversationBootstrap />
                </security>
                <textMessageEncoding maxReadPoolSize="64" maxWritePoolSize="16"
                    messageVersion="Soap11" writeEncoding="utf-8">
                    <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                        maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                </textMessageEncoding>
                <httpsTransport manualAddressing="false" maxBufferPoolSize="2147483647"
                    maxReceivedMessageSize="2147483647" allowCookies="false" authenticationScheme="Anonymous"
                    bypassProxyOnLocal="false" decompressionEnabled="true" hostNameComparisonMode="StrongWildcard"
                    keepAliveEnabled="true" maxBufferSize="2147483647" proxyAuthenticationScheme="Anonymous"
                    realm="" transferMode="Streamed" unsafeConnectionNtlmAuthentication="false"
                    useDefaultWebProxy="true" requireClientCertificate="false" />
            </binding>
        </customBinding>
    </bindings>
    <client>
        <endpoint address="https://secure.Some.com/API/1.2/Service.svc"
            binding="customBinding" bindingConfiguration="SomeService"
            contract="SomeService.SomeService" name="SomeService" />
    </client>
</system.serviceModel>

As you can see in the application configuration binding, I increased the maxBufferPoolSize, maxReceivedMessageSize, and maxBufferSize to 2147483647. I also changed the transferMode to Streamed.

You could be hitting a timeout issue (either at the client or server end)...how long does it take before the error occurs from the start of your request? It's more likely it is the server timing out and closing the connection because you asked for a large amount of data which takes too long to complete. If you don't have the 3rd party website under your control, then you can't change it's timings. Can you just batch 50 records at time? See this on timeout options. http://www.lybecker.com/blog/2010/10/14/wcf-timeouts/ — Colin Smith, Jul 31 '12 at 23:18
And if you are using Streaming you have to up the limits in IIS. http://stackoverflow.com/questions/6030137/large-binary-byte-file-transfer-through-wcf — Colin Smith, Jul 31 '12 at 23:27
I don't think it is a timeout issue since it fails within milliseconds of when the web service is called. If I request a smaller set of data, then it succeeds but it can take upwards of 1-2 seconds. — Registered User, Aug 06 '12 at 16:42

score 0 · Answer 1 · answered Jul 27 '12 at 20:48

There is another important setting that can be set by adding a behavior:

<behaviors>
  <endpointBehaviors>
    <behavior name="SomeBehavior">
      <dataContractSerializer maxItemsInObjectGraph="2147483647" />
    </behavior>
  </endpointBehaviors>
</behaviors>

Then, update your endpoint:

<client>
  <endpoint address="https://secure.Some.com/API/1.2/Service.svc"
     binding="customBinding" bindingConfiguration="SomeService"
     contract="SomeService.SomeService" name="SomeService"
     **behaviorConfiguration="SomeBehavior"**/>
</client>

Thanks for the advice. I tried this change, but I am still encountering the same error. — Registered User, Jul 27 '12 at 21:29

WCF SOAP Error for Larger Requests

1 Answers1