Authorization header is lost on redirect

Question

Below is the code that does authentication, generates the Authorization header, and calls the API.

Unfortunately, I get a 401 Unauthorized error following the GET request on the API.

However, when I capture the traffic in Fiddler and replay it, the call to the API is successful and I can see the desired 200 OK status code.

[Test]
public void RedirectTest()
{
    HttpResponseMessage response;
    var client = new HttpClient();
    using (var authString = new StringContent(@"{username: ""theUser"", password: ""password""}", Encoding.UTF8, "application/json"))
    {
        response = client.PostAsync("http://host/api/authenticate", authString).Result;
    }

    string result = response.Content.ReadAsStringAsync().Result;
    var authorization = JsonConvert.DeserializeObject<CustomAutorization>(result);
    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(authorization.Scheme, authorization.Token);
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.host+json;version=1");

    response =
        client.GetAsync("http://host/api/getSomething").Result;
    Assert.True(response.StatusCode == HttpStatusCode.OK);
}

When I run this code the Authorization header is lost.

However, in Fiddler that header is passed successfully.

Any idea what I'm doing wrong?

Answer 1

The reason you are experiencing this behavior is that it is by design.

Most HTTP clients (by default) strip out authorization headers when following a redirect.

One reason is security. The client could be redirected to an untrusted third party server, one that you would not want to disclose your authorization token to.

What you can do is detect that the redirect has occurred and reissue the request directly to the correct location.

Your API is returning 401 Unauthorized to indicate that the authorization header is missing (or incomplete). I will assume that the same API returns 403 Forbidden if the authorization information is present in the request but is simply incorrect (wrong username / password).

If this is the case, you can detect the 'redirect / missing authorization header' combination and resend the request.

---

Here is the code from the question rewritten to do this:

[Test]
public void RedirectTest()
{
    // These lines are not relevant to the problem, but are included for completeness.
    HttpResponseMessage response;
    var client = new HttpClient();
    using (var authString = new StringContent(@"{username: ""theUser"", password: ""password""}", Encoding.UTF8, "application/json"))
    {
        response = client.PostAsync("http://host/api/authenticate", authString).Result;
    }

    string result = response.Content.ReadAsStringAsync().Result;
    var authorization = JsonConvert.DeserializeObject<CustomAutorization>(result);

    // Relevant from this point on.
    client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(authorization.Scheme, authorization.Token);
    client.DefaultRequestHeaders.Add("Accept", "application/vnd.host+json;version=1");

    var requestUri = new Uri("http://host/api/getSomething");
    response = client.GetAsync(requestUri).Result;

    if (response.StatusCode == HttpStatusCode.Unauthorized)
    {
        // Authorization header has been set, but the server reports that it is missing.
        // It was probably stripped out due to a redirect.

        var finalRequestUri = response.RequestMessage.RequestUri; // contains the final location after following the redirect.

        if (finalRequestUri != requestUri) // detect that a redirect actually did occur.
        {
            if (IsHostTrusted(finalRequestUri)) // check that we can trust the host we were redirected to.
            {
               response = client.GetAsync(finalRequestUri).Result; // Reissue the request. The DefaultRequestHeaders configured on the client will be used, so we don't have to set them again.
            }
        }
    }

    Assert.True(response.StatusCode == HttpStatusCode.OK);
}


private bool IsHostTrusted(Uri uri)
{
    // Do whatever checks you need to do here
    // to make sure that the host
    // is trusted and you are happy to send it
    // your authorization token.

    if (uri.Host == "host")
    {
        return true;
    }

    return false;
}

---

Note that you could save the value of finalRequestUri and use it for future requests to avoid the extra request involved in the retry. However as this is a temporary redirect you should probably issue the request to the original location each time.

Answer 2

I would turn off the automatic redirect behavior and create a client hander that hides the code dealing with the temporary redirect. The HttpClient class allows you to install DelegatingHandlers from which you can modify the request of response.

public class TemporaryRedirectHandler : DelegatingHandler
{
    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        var response = await base.SendAsync(request, cancellationToken);
        if (response.StatusCode == HttpStatusCode.TemporaryRedirect)
        {
            var location = response.Headers.Location;
            if (location == null)
            {
                return response;
            }

            using (var clone = await CloneRequest(request, location))
            {
                response = await base.SendAsync(clone, cancellationToken);
            }
        }
        return response;
    }


    private async Task<HttpRequestMessage> CloneRequest(HttpRequestMessage request, Uri location)
    {
        var clone = new HttpRequestMessage(request.Method, location);

        if (request.Content != null)
        {
            clone.Content = await CloneContent(request);
            if (request.Content.Headers != null)
            {
                CloneHeaders(clone, request);
            }
        }

        clone.Version = request.Version;
        CloneProperties(clone, request);
        CloneKeyValuePairs(clone, request);
        return clone;
    }

    private async Task<StreamContent> CloneContent(HttpRequestMessage request)
    {
        var memstrm = new MemoryStream();
        await request.Content.CopyToAsync(memstrm).ConfigureAwait(false);
        memstrm.Position = 0;
        return new StreamContent(memstrm);
    }

    private void CloneHeaders(HttpRequestMessage clone, HttpRequestMessage request)
    {
        foreach (var header in request.Content.Headers)
        {
            clone.Content.Headers.Add(header.Key, header.Value);
        }
    }

    private void CloneProperties(HttpRequestMessage clone, HttpRequestMessage request)
    {
        foreach (KeyValuePair<string, object> prop in request.Properties)
        {
            clone.Properties.Add(prop);
        }
    }

    private void CloneKeyValuePairs(HttpRequestMessage clone, HttpRequestMessage request)
    {
        foreach (KeyValuePair<string, IEnumerable<string>> header in request.Headers)
        {
            clone.Headers.TryAddWithoutValidation(header.Key, header.Value);
        }
    }
}

You would instantiate the HttpClient like this:

var handler = new TemporaryRedirectHandler()
{
    InnerHandler = new HttpClientHandler()
    {
        AllowAutoRedirect = false
    }
};

HttpClient client = new HttpClient(handler);

Answer 3

I had a similar problem, but not quite the same. In my case, I also had the redirect problem, but security is implemented with OAuth, which also has the secondary, but related, problem that tokens sometimes expire.

For that reason, I'd like to be able to configure an HttpClient to automatically go and refresh the OAuth token when it receives a 401 Unauthorized response, regardless of whether this happens because of a redirect, or a token expiration.

The solution posted by Chris O'Neill shows the general steps to take, but I wanted to embed that behaviour inside of an HttpClient object, instead of having to surround all our HTTP code with an imperative check. We have a lot of existing code that uses a shared HttpClient object, so it'd be much easier to refactor our code if I could change the behaviour of that object.

The following looks like it's working. I've only prototyped it so far, but it seems to be working. Much of our code base is in F#, so the code is in F#:

open System.Net
open System.Net.Http

type TokenRefresher (refreshAuth, inner) =
    inherit MessageProcessingHandler (inner)

    override __.ProcessRequest (request, _) = request

    override __.ProcessResponse (response, cancellationToken) =
        if response.StatusCode <> HttpStatusCode.Unauthorized
        then response
        else
            response.RequestMessage.Headers.Authorization <- refreshAuth ()
            inner.SendAsync(response.RequestMessage, cancellationToken).Result

This is a little class that takes care of refreshing the Authorization header if it receives a 401 Unauthorized response. It refreshes using an injected refreshAuth function, which has the type unit -> Headers.AuthenticationHeaderValue.

Since this is still prototype code, I made the inner SendAsync call a blocking call, thereby leaving it as an exercise to the reader to implement it properly using an async workflow.

Given a refresh function called refreshAuth, you can create a new HttpClient object like this:

let client = new HttpClient(new TokenRefresher(refreshAuth, new HttpClientHandler ()))

The answer posted by Chris O'Neill takes care to check that the new URL is still considered safe. I skipped that security consideration here, but you should strongly consider including a similar check before retrying the request.

Answer 4

Loop Requests worked for me (python)

response = do_request(url, access_tok, "GET", payload={}, headers={}, allow_redirect=False)
    
if response.status_code in range(300, 310):
        new_response = do_request(response.headers['Location'], access_tok, "GET", payload={}, headers={},)
        # print(new_response.status_code)
        pprint(new_response.json())
        # print(new_response.url)