Can't use GetStringAsync() if I want to change the Referer header each time?

Question

It seems that HttpClient is the recommended way to do HTTP communication. Downloading HTML of a URL seemed easy like

var html = httpClient.GetStringAsync(url);

But I need to change some header values like Referer each time. At first, I tried

httpClient.DefaultRequestHeaders.Add("Referer", referrer);

, but this caused an error at the second time. It seems that it is once set, cannot be changed.

I searched for a solution and found one ( https://stackoverflow.com/a/12023307/455796 ), but this seems very complicated than GetStringAsync. I need to create a HttpRequestMessage, call SendAsync, continue to call response.Content.ReadAsAsync, call Wait(), and then read the result. Also, the comment says I need to dispose the HttpRequestMessage. If this is the only way to change headers, I will do so, but is this the best way? Can't I use GetStringAsync and change a header value?

You can't `Add` another referer value without removing the previous one, the error you are seeing is that there is already one and can't be multiple ones. So add a check to remove if it if exists already, then add your new value. — Karl-Johan Sjögren, Feb 08 '21 at 22:44

Andy · Accepted Answer · 2021-02-09T01:49:04.053

You could do it this way:

using(var msg = new HttpRequestMessage(HttpMethod.Get, new Uri("https://www.test.com")))
{
    msg.Headers.Referrer = new Uri("https://www.example.com");
    using (var req = await _httpClient.SendAsync(msg))
    {
        var str1 = await req.Content.ReadAsStringAsync();
    }
}

or, if you want to override the default request headers, do it this way:

_httpClient.DefaultRequestHeaders.Referrer = new Uri("https://www.example.com");
var str2 = await _httpClient.GetStringAsync("https://www.test.com/");

The first way is if you want to reuse your HttpClient throughout the lifetime of your application. Meaning, if you share it with every object or method that needs to make an HTTP request. Doing it this will not cause conflicts if multiple threads are trying to use it at the same time when each thread is modifying headers. DeafaultRequestHeaders is not thread-safe... so you could get exceptions thrown if more than one thread is modifying the DeafaultRequestHeaders property at the same time.

Can't use GetStringAsync() if I want to change the Referer header each time?

1 Answers1