HTTPWebRequest & After a while, 400 (Bad Request) starts

Question

I have a background worker thread that is constantly syncing data to/from a remote server. When I first run the app, everything appears to work fine. It makes the web request about every 30 seconds and the data returns successfully.

If I leave the simulator running for a long time, eventually the request will fail with a (400) Bad Request. And all subsequent requests do the same thing. If I kill the app and restart it...all is well.

Anyone have any ideas? Code is below.

public RestResponse<T> Execute<T>(RestRequest request) {
    var restResponse = new RestResponse<T>();
    var serializer = new JavaScriptSerializer();
    var urlPath = baseUrl + "/" + request.Resource;
    Console.WriteLine("Requesting: " + urlPath);
    var httpRequest = (HttpWebRequest)HttpWebRequest.Create(new Uri(urlPath));

    httpRequest.Headers = request.Headers;
    foreach (string key in clientHeaders.Keys)
        httpRequest.Headers.Add(key, clientHeaders[key]);
    httpRequest.Headers.Add("Accept-Encoding", "gzip,deflate");

    Authenticator.Authenticate(httpRequest);
    httpRequest.Method = request.Method.ToString();       
    HttpWebResponse httpResponse = null;
    try {
        if ((request.Method == Method.POST) && (!request.IsJsonPost))
            SetPostData(httpRequest, request);

        if ((request.Method == Method.POST) && (request.IsJsonPost)){
            SetJsonPostData(httpRequest, request);
        }

        httpResponse = (HttpWebResponse)httpRequest.GetResponse();
        var reader = new StreamReader(GetStreamForResponse(httpResponse));
        var responseString = reader.ReadToEnd();
        Console.WriteLine(responseString);
        reader.Close();
        restResponse.StatusCode = httpResponse.StatusCode;
        restResponse.Headers = httpResponse.Headers;
        restResponse.Data = serializer.Deserialize<T>(responseString);
        restResponse.ResponseStatus = ResponseStatus.Completed;
        httpResponse.Close();
    } catch (WebException e) {
        restResponse.ResponseStatus = ResponseStatus.Error;
        restResponse.ErrorMessage = e.Message;
        restResponse.ErrorException = e;
        var webResponse = (HttpWebResponse)e.Response;
        if (webResponse != null) {
            restResponse.StatusCode = webResponse.StatusCode;
            restResponse.Headers = webResponse.Headers;
        }
        if (restResponse.StatusCode != HttpStatusCode.NotModified)
            Console.WriteLine("An exception occured:\r\n " + request.Resource + "\r\n" + e + "\r\n");
    } catch (Exception ex) {
        restResponse.ResponseStatus = ResponseStatus.Error;
        restResponse.ErrorMessage = ex.Message;
        restResponse.ErrorException = ex;
    }

    if (httpResponse != null)
        httpResponse.Close();

    return restResponse;
}

It fails on this line:

httpResponse = (HttpWebResponse)httpRequest.GetResponse();

Stack Trace:

System.Net.WebException: The remote server returned an error: (400) Bad Request.
  at System.Net.HttpWebRequest.CheckFinalStatus (System.Net.WebAsyncResult result) [0x002f2] in /Developer/MonoTouch/Source/mono/mcs/class/System/System.Net/HttpWebRequest.cs:1477 
  at System.Net.HttpWebRequest.SetResponseData (System.Net.WebConnectionData data) [0x00141] in /Developer/MonoTouch/Source/mono/mcs/class/System/System.Net/HttpWebRequest.cs:1300

The request is never getting to the remote server.

The **400** comes from the server. It would be helpful if you could run a sniffer (e.g. wireshark) and find out what was transmitted in that case (and compare it to the normal response you got before that). — poupou, Jun 08 '12 at 00:56
That's the weird thing, its not coming from the server. IIS never sees the request come in. — Chris Kooken, Jun 08 '12 at 01:55
Any reason to why your request is instantiated as like that? I would prefer `(HttpWebRequest)WebRequest.Create(new Uri(urlPath));`. Have you tried using your code on a service that does not need authentication? Have you tried not setting the Post data? — Cheesebaron, Jun 12 '12 at 09:24
Are there some long running requests (takes a long time for the server to respond)? When this issue "is on", what happens if you restart the remote server instead of you own app? Do you have the possibility to try that? The reason I'm asking is that I think the OS prevents too many simultaneous outgoing request to the same destination... — user1429080, Jun 26 '12 at 07:40
Just a guess, but there could be a regression in HttpWebRequest that leads to the same behavior as described in this bug: https://bugzilla.novell.com/show_bug.cgi?id=648862 So just for the sake of it try to set `httpRequest.KeepAlive = false;` and see if that helps. Otherwise I would recommend creating a new bug report in xamarins bugzilla and provide a test case which can be reproduced on their end. Another far-fetched suggestion could be to try make you code async, but I don't see why that should behave differently. — Cheesebaron, Jun 26 '12 at 17:41
Do you have an escaped / character (%2f) in a URI (request.Resource)? Or does it appear after time? There is a problem with Uri constructor: http://stackoverflow.com/questions/10413795/c-sharp-webrequest-getresponse-400-bad-request — Ivan Nikitin, Jun 28 '12 at 13:12

score 2 · Accepted Answer · answered Jun 28 '12 at 15:02

2

The problem was in the clientHeaders collection. This RESTClient class is instantiated once during the applicaiton life cycle. I was adding headers over and over and not clearing them out first. After a while, the headers got too big and a bad request was issued.

answered Jun 28 '12 at 15:02

Chris Kooken

32,730
15
85
123

1

It wasn't easy to understand this by looking at the code provided:) – Ivan Nikitin Jun 30 '12 at 10:16

NickD · Answer 2 · 2012-06-23T14:15:25.830

0

I assume you run out of connections. You can try to increase them:

<configuration> <system.net> <connectionManagement> <add address="*" maxconnection="65535" /> </connectionManagement> </system.net> </configuration>

edited Jun 23 '12 at 14:15

answered Jun 23 '12 at 14:08

NickD

2,672
7
37
58

you could try anyway to increase the connections. – NickD Jun 23 '12 at 14:15
How do I do that in MonoTouch? – Chris Kooken Jun 23 '12 at 14:16
System.Net.ServicePointManager.DefaultConnectionLimit = 65535; – NickD Jun 23 '12 at 14:24
i just look in the system.web.dll using reflector what can cause a http 400. – NickD Jun 23 '12 at 14:26
if (this.QueryStringText.Length > httpRuntime.MaxQueryStringLength) { throw new HttpException(400, System.Web.SR.GetString("QueryString_too_long")); } – NickD Jun 23 '12 at 14:32
Not the querystring. My URL is very short. – Chris Kooken Jun 23 '12 at 16:00
hmm, do you modify baseUrl? maybe it is related to threading? what happens if you run 1000 requests serial? – NickD Jun 23 '12 at 16:31

HTTPWebRequest & After a while, 400 (Bad Request) starts

2 Answers2