Request headers must contain only ASCII characters while using c# http client?

Question

I am using HttpClient to make a POST call by passing header but at some point of time I am getting an error as:

Request headers must contain only ASCII characters.

With stacktrace as:

at System.Net.Http.HttpConnection.WriteStringAsync(String s)
   at System.Net.Http.HttpConnection.WriteHeadersAsync(HttpHeaders headers, String cookiesFromContainer)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)

Here is my code:

public HttpWrapper(string endpoint, Func<IDictionary<string, string>> NewHeader)
{
    _httpClient = new HttpClient();
    _httpClient.BaseAddress = new Uri(endpoint);

    if (NewHeader != null)
    {
        var headers = NewHeader();
        foreach (var header in headers)
        {
            _httpClient.DefaultRequestHeaders.TryAddWithoutValidation(header.Key, header.Value);
        }
    }
}

Do I need to do something with header.Value to fix this issue? I was reading online so looks like I need to use Utf-8 here but not sure on how to do it properly?

Update

I got header value like this today and it threw same exception since HttpUtility.HtmlEncode didn't do anything on it. Also I am not sure what is this character <0x94>? Any thoughts why it is happening?

Also I am not sure

"In order to use non ASCII characters in URI you need to escape them using the %hexcode syntax (see section 2 of RFC 2396)." from https://stackoverflow.com/a/5251951/5386938 — , Nov 11 '20 at 18:44
Here's [one way](https://stackoverflow.com/a/34189188/5386938) — , Nov 11 '20 at 19:50
What about the accepted answer? Is that correct way as well? @JustinEzequiel — AndyP, Nov 11 '20 at 19:55

Yigit Yuksel · Accepted Answer · 2020-11-11T18:59:39.143

3

You can try to encode header values to HTML:

_httpClient.DefaultRequestHeaders.TryAddWithoutValidation(header.Key, HttpUtility.HtmlEncode(header.Value));

Your string, Ergänzendes will be "Ergänzendes".

edited Nov 11 '20 at 18:59

answered Nov 11 '20 at 18:39

Yigit Yuksel

1,060
2
18
35

I tried like this already but it doesn't give me that value - `string str = "Ergänzendes"; string val = Convert.ToBase64String(Encoding.UTF8.GetBytes(str)); Console.WriteLine(val);` back. Is this the right expectation? It just prints this value `RXJnw6RuemVuZGVz` – AndyP Nov 11 '20 at 18:41
So internally `httpclient` will convert this base64 encoded string `RXJnw6RuemVuZGVz` to the way it came back originally? Is my assumptions correct here? – AndyP Nov 11 '20 at 18:47
Hmm, it's changing the original text. You can try to encode to HMTL your string. The answer is updated. – Yigit Yuksel Nov 11 '20 at 18:59
It then prints like this - `Ergänzendes`. I am not sure if that is the right way? – AndyP Nov 11 '20 at 19:03
Yes, this is the correct way. The browsers render the header text to "Ergänzendes ". – Yigit Yuksel Nov 11 '20 at 19:04
In my case I am calling a different service using `httpclient` by passing these headers so do you think this is the right way for the fix? They should be able to encode this back properly? – AndyP Nov 11 '20 at 19:09
They can decode the data into "Ergänzendes " without any problem using HttpUtility.HtmlDecode("Ergänzendes"). You can't send non ASCII chars through headers thus, you have to decode to html. – Yigit Yuksel Nov 11 '20 at 19:14
Maybe other headers have the same problem? – Yigit Yuksel Nov 11 '20 at 20:44
yeah nvm I got that fixed. Sorry for late reply on that. I got one very weird problem now. For some reason today we saw a string coming as like I posted in my question and `HttpUtility.HtmlEncode` didn't do anything on that and then later on my HttpClient threw exception with same `Request headers must contain only ASCII characters.` Any thoughts why it could be and how we can solve it? – AndyP Feb 12 '21 at 01:41
It looks like it's a cancel character but not sure why `HttpUtility.HtmlEncode` didn't do anything on it? – AndyP Feb 12 '21 at 03:29

score 2 · Answer 2 · answered Nov 16 '20 at 17:21

2

I hope this may help in your specific scenario or others with a similar situation. In my case, my header was containing non ASCII characters as one of the Headers values was generated by creating a sha256 HMAC. That was a requirement from the service I was trying to send my request. So, what I had to do in order to encode those characters into ASCII was just this:

//hashedSignature is the one containing NON ASCII charcters!
string MessageSignatureValue = System.Text.Encoding.ASCII.GetString(hashedSignature); 
httpClient.DefaultRequestHeaders.Add(MessageSignaturField, MessageSignatureValue);

I have just included the importants pieces of code.

So, System.Text.Encoding.ASCII.GetString() method did the trick!

Again, I hope it helps.

answered Nov 16 '20 at 17:21

Sebastian Inones

1,561
1
19
32

I see. So what's the difference between accepted answer and your suggestion? Any thoughts? – AndyP Nov 16 '20 at 17:23
You sure this is the right syntax - `string MessageSignatureValue = System.Text.Encoding.ASCII.GetString(hashedSignature); `? – AndyP Nov 16 '20 at 17:28
Honestly, I have never used **HttpUtility**. So, I haven't a concrete answer about that. But, from the documentation it states: "Converts a string into an *HTML-encoded* string." If that works for you there will be no difference, except the in the way I call the method it's telling that you want something **encoded** to ASCII explicitely. Maybe, it's more esay to understand the purpose. Just a thought. Here you have the [link to the ASCIIEncoding](https://learn.microsoft.com/en-us/dotnet/api/system.text.asciiencoding.getstring?view=netcore-3.1) – Sebastian Inones Nov 16 '20 at 17:31
This answer will convert non-recognised characters to '?' so the outputted header value or name, will have a '?' somewhere in the char sequence where the non-ascii char was detected. This might not be the answer some devs are looking for,. – IbrarMumtaz Aug 03 '23 at 15:30

Request headers must contain only ASCII characters while using c# http client?

2 Answers2