How to add encoding information to the response stream in ASP.NET?

Question

I have following piece of code:

public void ProcessRequest (HttpContext context) 
{
    context.Response.ContentType = "text/rtf; charset=UTF-8";
    context.Response.Charset = "UTF-8";
    context.Response.ContentEncoding = System.Text.Encoding.UTF8;
    context.Response.AddHeader("Content-disposition", "attachment;filename=lista_obecnosci.csv");
    context.Response.Write("ąęćżźńółĄŚŻŹĆŃŁÓĘ");
}

When I try to open generated csv file, I get following behavior:

In Notepad2 - everything is fine.
In Word - conversion wizard opens and asks to convert the text. It suggest UTF-8, which is somehow ok.
In Excel - I get real mess. None of those Polish characters can be displayed.

I wanted to write those special encoding-information characters in front of my string, i.e.

context.Response.Write((char)0xef);
context.Response.Write((char)0xbb);
context.Response.Write((char)0xbf);

but that won't do any good. The response stream is treating that as normal data and converts it to something different.

I'd appreciate help on this one.

score 26 · Answer 1 · answered Sep 18 '12 at 20:00

26

I ran into the same problem, and this was my solution:

context.Response.BinaryWrite(System.Text.Encoding.UTF8.GetPreamble());
context.Response.Write("ąęćżźńółĄŚŻŹĆŃŁÓĘ");

answered Sep 18 '12 at 20:00

Collin K

15,277
1
27
22

I wonder, is this roughly the same as what happens in Alan Moores's answer? – Kai Hartmann Jan 31 '14 at 11:01
1

this is most useful answer, since every encoding and byte order system has a different preamble and this applies to other encodings. – Garr Godfrey Nov 03 '16 at 15:54
This was driving me nuts. I was sending Hebrew text in Unicode and you could see it in Notepad and Notepad++ but Excel/Word/Wordpad all showed gibberish. But your answer fixed it. – adinas Jan 24 '20 at 09:06

Alan Moore · Accepted Answer · 2013-10-29T00:57:01.860

What you call "encoding-information" is actually a BOM. I suspect each of those "characters" is getting encoded separately. To write the BOM manually, you have to write it as three bytes, not three characters. I'm not familiar with the .NET I/O classes, but there should be a method available to you that takes a byte or byte[] parameter and writes them directly to the file.

By the way, the UTF-8 BOM is optional; in fact, its use is discouraged by the Unicode Consortium. If you don't have a specific reason for using it, save yourself some hassle and leave it out.

EDIT: I just remembered you can also write the actual BOM character, '\uFEFF', and let the encoder handle it:

context.Response.Write('\uFEFF');

thanks a lot! that exactly what i've been looking for. the purpose of the aphx handles is purely to generate excell friendly list, and this does the trick! — Greg, Jun 18 '09 at 07:02

score 2 · Answer 3 · edited May 23 '17 at 12:02

2

I think the problem is with Excel based on Microsoft Excel mangles Diacritics in .csv files. To prove this, copy your sample output string of ąęćżźńółĄŚŻŹĆŃŁÓĘ and paste into a test file using your favorite editor, and save as a UTF-8 encoded .csv file. Open in Excel and see the same issues.

edited May 23 '17 at 12:02

Community

1
1

answered Jun 18 '09 at 01:51

Kevin Hakanson

41,386
23
126
155

score 1 · Answer 4 · edited May 23 '17 at 10:30

1

The answer from Alan Moore translated to VB:

Context.Response.Write(""c)

edited May 23 '17 at 10:30

Community

1
1

answered Apr 17 '12 at 13:22

Manuel Alves

3,885
2
30
24

can you please explain this syntax? I'm not familiar with it – MtwStark Nov 28 '18 at 15:05
1

It's to make sure that what's on the left is treated as char and not as a string. Please see https://stackoverflow.com/a/19522767/251674 – Manuel Alves Nov 28 '18 at 16:58

How to add encoding information to the response stream in ASP.NET?

4 Answers4

Linked