Universal app HttpClient header: how to disable some of the headers?

Question

A bit of background: I'm trying to "port" an android app to Windows Phone that calls a non-open web API. Since the API is not open or documented, I used Fiddler, run the app's android version, and snooped the API calls it made.

I'm using Windows.Web.Http.HttpClient as the class of choice since it seems like this will be the class moving on instead of System.Net.Http.HttpClient.

Here's the C# code excerpt that I use to generate an HTTP POST request:

HttpBaseProtocolFilter _httpFilter = new HttpBaseProtocolFilter();
HttpClient _httpClient = new HttpClient(_httpFilter);                    
_httpClient.DefaultRequestHeaders.AcceptEncoding.Clear();
_httpClient.DefaultRequestHeaders.Accept.TryParseAdd("application/xml");
_httpClient.DefaultRequestHeaders.AcceptLanguage.TryParseAdd("en");            
_httpClient.DefaultRequestHeaders.Connection.TryParseAdd("Keep-Alive");
_httpClient.DefaultRequestHeaders.Add("message-version", "1");
_httpClient.DefaultRequestHeaders.UserAgent.TryParseAdd("Android|SAMSUNG-  SGH-I337|3.3.1"); 
_httpClient.DefaultRequestHeaders.Cookie.TryParseAdd(cookie); //Some cookie values                                

Uri uri = new Uri(SOMEURI);
XDocument xd = new XDocument(STUFF_TO_BUILD_XML);
string xd_str = string.Concat(xd.Declaration.ToString(), xd.ToString());
xd_str = xd_str.Replace("\r\n", string.Empty);
xd_str = xd_str.Replace("  ", string.Empty); 
HttpRequestMessage req_msg = new HttpRequestMessage(HttpMethod.Post, uri);
HttpStringContent strcnt = new HttpStringContent(xd_str);            
req_msg.Content = strcnt;
req_msg.Content.Headers.ContentType = new     Windows.Web.Http.Headers.HttpMediaTypeHeaderValue("text/xml; charset=UTF-8");
req_msg.Headers.Host = new Windows.Networking.HostName(SOMEHOSTNAME);

HttpResponseMessage rsp_msg = await _httpClient.SendRequestAsync(req_msg);

Here's the raw text Fiddler sees when making the API call using my code:

POST <HTTPS endpoint> HTTP/1.1
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Host: <hostname>
Cookie2: Version=1
Accept: application/xml
message-version: 1
User-Agent: Android|SAMSUNG-SGH-I337|3.3.1
Accept-Language: en
Content-Length: 173
Content-Type: text/xml; charset=UTF-8
Cache-Control: no-cache
Cookie: STR1=VAL1; STR2=VAL2

<MESSAGE_IN_XML>

--Response--
HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.5; JBoss-5.0/JBossWeb-2.1
X-Frame-Options: SAMEORIGIN
Transfer-Encoding: chunked
Date: Fri, 03 Apr 2015 01:18:07 GMT

0

Here's the raw text Fiddler sees when making request via Android app:

POST <HTTPS endpoint> HTTP/1.1
Content-Type: text/xml; charset=UTF-8
Connection: Keep-Alive
accept: application/xml
user-agent: Android|SAMSUNG-SGH-I337|3.4
message-version: 1
Accept-Language: en
Content-Length: 173
Host: <hostname>
Cookie: STR1=VAL1; STR2=VAL2
Cookie2: $Version=1

<MESSAGE_IN_XML>

--response--
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.5; JBoss-5.0/JBossWeb-2.1
X-Frame-Options: SAMEORIGIN
Content-Type: application/xml;charset=utf-8
Date: Fri, 03 Apr 2015 01:08:22 GMT
Content-Length: 364

<MESSAGE_IN_XML>

See, from Fiddler's output, the only difference I see is in the header, the Accept-Encoding and Cache-Control entries. Is there a way to NOT send them? Or am I missing something here?

Answer 1

You are supposed to send the Authorization header.

Sample: Authorization: Basic àaaaaaaa

That will resolve the unauthorized issue.

Explanation:

We have several ways to secure the service's that are made available to the public. The most commonly used one is to pass on the credentials from the client application to the target application via authorization headers.

The authorization headers are added to the request by the client. In C#, we typically use AuthenticationHeaderValue

Samples can be found here.

http://leastprivilege.com/2012/03/14/asp-net-webapi-security-4-examples-for-various-authentication-scenarios/

Answer 2

I agree with Saravanan's analysis, the problem "appears" to be related to Authorization of the request, however, what you asked is how to disable some of the headers (I'm not going to make a judgment call on whether this is "correct" without more context as to what you're doing), so...

You can "fudge" the HttpClient class by using inheritance to create your own version, given that I can't see any code within System.Net.Http.HttpClient itself (even in the .NET Reference source, strangely) I had to create my own version using some trickery (instantiating the base class object and then assigning values using a second custom class created based off the sealed HttpRequestHeaders class).

Add this class to your project and you'll then want to replace references of HttpClient in your code with MyHttpClient... The following is a class you can use to remove those headers (Accept-Encoding and Cache-Control), I've tested in Fiddler to double-check it works:

public class MyHttpClient : HttpClient
{        

    public MyHttpClient(HttpClientHandler handler) : base(handler) {
        DefaultRequestHeaders = new MyHttpRequestHeaders();
    }

    //
    // Summary:
    //     Gets or Sets the headers which should be sent with each request.
    //
    // Returns:
    //     Returns The headers which should
    //     be sent with each request.
    public new MyHttpRequestHeaders DefaultRequestHeaders { get; set; }
}

public class MyHttpRequestHeaders : HttpHeaders
{
    public MyHttpRequestHeaders()
    {
        HttpClient client = new HttpClient();
        this.Accept = client.DefaultRequestHeaders.Accept;
        this.AcceptCharset = client.DefaultRequestHeaders.AcceptCharset;
        this.AcceptLanguage = client.DefaultRequestHeaders.AcceptLanguage;
        this.Authorization = client.DefaultRequestHeaders.Authorization;
        this.Connection = client.DefaultRequestHeaders.Connection;
        this.ConnectionClose = client.DefaultRequestHeaders.ConnectionClose;
        this.Date = client.DefaultRequestHeaders.Date;
        this.Expect = client.DefaultRequestHeaders.Expect;
        this.ExpectContinue = client.DefaultRequestHeaders.ExpectContinue;
        this.From = client.DefaultRequestHeaders.From;
        this.Host = client.DefaultRequestHeaders.Host;
        this.IfMatch = client.DefaultRequestHeaders.IfMatch;
        this.IfModifiedSince = client.DefaultRequestHeaders.IfModifiedSince;
        this.IfNoneMatch = client.DefaultRequestHeaders.IfNoneMatch;
        this.IfRange = client.DefaultRequestHeaders.IfRange;
        this.IfUnmodifiedSince = client.DefaultRequestHeaders.IfUnmodifiedSince;
        this.MaxForwards = client.DefaultRequestHeaders.MaxForwards;
        this.Pragma = client.DefaultRequestHeaders.Pragma;
        this.ProxyAuthorization = client.DefaultRequestHeaders.ProxyAuthorization;
        this.Range = client.DefaultRequestHeaders.Range;
        this.Referrer = client.DefaultRequestHeaders.Referrer;
        this.TE = client.DefaultRequestHeaders.TE;
        this.Trailer = client.DefaultRequestHeaders.Trailer;
        this.TransferEncoding = client.DefaultRequestHeaders.TransferEncoding;
        this.TransferEncodingChunked = client.DefaultRequestHeaders.TransferEncodingChunked;
        this.Upgrade = client.DefaultRequestHeaders.Upgrade;
        this.UserAgent = client.DefaultRequestHeaders.UserAgent;
        this.Via = client.DefaultRequestHeaders.Via;
        this.Warning = client.DefaultRequestHeaders.Warning;
    }

    // Summary:
    //     Gets the value of the Accept header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Accept header for an HTTP request.
    public HttpHeaderValueCollection<MediaTypeWithQualityHeaderValue> Accept { get; set; }
    //
    // Summary:
    //     Gets the value of the Accept-Charset header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Accept-Charset header for an HTTP request.
    public HttpHeaderValueCollection<StringWithQualityHeaderValue> AcceptCharset { get; set; }
    //
    // Summary:
    //     Gets the value of the Accept-Language header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Accept-Language header for an HTTP request.
    public HttpHeaderValueCollection<StringWithQualityHeaderValue> AcceptLanguage { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the Authorization header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.AuthenticationHeaderValue.The value of the
    //     Authorization header for an HTTP request.
    public AuthenticationHeaderValue Authorization { get; set; }
    //
    // Summary:
    //     Gets the value of the Connection header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Connection header for an HTTP request.
    public HttpHeaderValueCollection<string> Connection { get; set; }
    //
    // Summary:
    //     Gets or sets a value that indicates if the Connection header for an HTTP
    //     request contains Close.
    //
    // Returns:
    //     Returns System.Boolean.true if the Connection header contains Close, otherwise
    //     false.
    public bool? ConnectionClose { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the Date header for an HTTP request.
    //
    // Returns:
    //     Returns System.DateTimeOffset.The value of the Date header for an HTTP request.
    public DateTimeOffset? Date { get; set; }
    //
    // Summary:
    //     Gets the value of the Expect header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Expect header for an HTTP request.
    public HttpHeaderValueCollection<NameValueWithParametersHeaderValue> Expect { get; set; }
    //
    // Summary:
    //     Gets or sets a value that indicates if the Expect header for an HTTP request
    //     contains Continue.
    //
    // Returns:
    //     Returns System.Boolean.true if the Expect header contains Continue, otherwise
    //     false.
    public bool? ExpectContinue { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the From header for an HTTP request.
    //
    // Returns:
    //     Returns System.String.The value of the From header for an HTTP request.
    public string From { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the Host header for an HTTP request.
    //
    // Returns:
    //     Returns System.String.The value of the Host header for an HTTP request.
    public string Host { get; set; }
    //
    // Summary:
    //     Gets the value of the If-Match header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the If-Match header for an HTTP request.
    public HttpHeaderValueCollection<EntityTagHeaderValue> IfMatch { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the If-Modified-Since header for an HTTP request.
    //
    // Returns:
    //     Returns System.DateTimeOffset.The value of the If-Modified-Since header for
    //     an HTTP request.
    public DateTimeOffset? IfModifiedSince { get; set; }
    //
    // Summary:
    //     Gets the value of the If-None-Match header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.Gets the value
    //     of the If-None-Match header for an HTTP request.
    public HttpHeaderValueCollection<EntityTagHeaderValue> IfNoneMatch { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the If-Range header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.RangeConditionHeaderValue.The value of the
    //     If-Range header for an HTTP request.
    public RangeConditionHeaderValue IfRange { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the If-Unmodified-Since header for an HTTP request.
    //
    // Returns:
    //     Returns System.DateTimeOffset.The value of the If-Unmodified-Since header
    //     for an HTTP request.
    public DateTimeOffset? IfUnmodifiedSince { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the Max-Forwards header for an HTTP request.
    //
    // Returns:
    //     Returns System.Int32.The value of the Max-Forwards header for an HTTP request.
    public int? MaxForwards { get; set; }
    //
    // Summary:
    //     Gets the value of the Pragma header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Pragma header for an HTTP request.
    public HttpHeaderValueCollection<NameValueHeaderValue> Pragma { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the Proxy-Authorization header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.AuthenticationHeaderValue.The value of the
    //     Proxy-Authorization header for an HTTP request.
    public AuthenticationHeaderValue ProxyAuthorization { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the Range header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.RangeHeaderValue.The value of the Range header
    //     for an HTTP request.
    public RangeHeaderValue Range { get; set; }
    //
    // Summary:
    //     Gets or sets the value of the Referer header for an HTTP request.
    //
    // Returns:
    //     Returns System.Uri.The value of the Referer header for an HTTP request.
    public Uri Referrer { get; set; }
    //
    // Summary:
    //     Gets the value of the TE header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the TE header for an HTTP request.
    public HttpHeaderValueCollection<TransferCodingWithQualityHeaderValue> TE { get; set; }
    //
    // Summary:
    //     Gets the value of the Trailer header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Trailer header for an HTTP request.
    public HttpHeaderValueCollection<string> Trailer { get; set; }
    //
    // Summary:
    //     Gets the value of the Transfer-Encoding header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Transfer-Encoding header for an HTTP request.
    public HttpHeaderValueCollection<TransferCodingHeaderValue> TransferEncoding { get; set; }
    //
    // Summary:
    //     Gets or sets a value that indicates if the Transfer-Encoding header for an
    //     HTTP request contains chunked.
    //
    // Returns:
    //     Returns System.Boolean.true if the Transfer-Encoding header contains chunked,
    //     otherwise false.
    public bool? TransferEncodingChunked { get; set; }
    //
    // Summary:
    //     Gets the value of the Upgrade header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Upgrade header for an HTTP request.
    public HttpHeaderValueCollection<ProductHeaderValue> Upgrade { get; set; }
    //
    // Summary:
    //     Gets the value of the User-Agent header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the User-Agent header for an HTTP request.
    public HttpHeaderValueCollection<ProductInfoHeaderValue> UserAgent { get; set; }
    //
    // Summary:
    //     Gets the value of the Via header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Via header for an HTTP request.
    public HttpHeaderValueCollection<ViaHeaderValue> Via { get; set; }
    //
    // Summary:
    //     Gets the value of the Warning header for an HTTP request.
    //
    // Returns:
    //     Returns System.Net.Http.Headers.HttpHeaderValueCollection<T>.The value of
    //     the Warning header for an HTTP request.
    public HttpHeaderValueCollection<WarningHeaderValue> Warning { get; set; }
}