How to make sure my created filedownload is UTF-8? (and not UTF-8 without BOM)

Question

i've made a download function to download messages to a CSV file (code is below). Now when i open it up in notepad or notepad++ i see this:

é NY ø ╬ ║► ░ ê ö

(and that is what is in the database btw)

Now, when i open it up in Ms-Excel it shows this:

Ã© NY Ã¸ â•¬ â•‘â–º â–‘ Ãª Ã¶

When i open it up in notepad++, it says it's encoded in 'UTF8 without BOM'. When i encode it (in notepad++) to UTF-8, all goes well (that is, Excel shows the right chars too)

But how can i make sure that the file i create from my code is UTF-8?

This is my code:

public ActionResult DownloadPersonalMessages()
{    
    StringBuilder myCsv = new StringBuilder();
    myCsv.Append(new DownloadService().GetPersonalMessages());

    this.Response.ContentType = "text/csv";
    Response.AddHeader("content-disposition", "attachment; filename=PersonalMessages.csv");
    Response.ContentEncoding = Encoding.UTF8;
    Response.Write(myCsv.ToString());
    Response.Flush();
    Response.HeaderEncoding = Encoding.UTF8;
    return Content("");
}

Edit:

my function now returns a ByteArray with this conversion

UTF8Encoding encoding = new UTF8Encoding();
return encoding.GetBytes(str);

and my download is now this:

Response.AddHeader("Content-Disposition", "attachment; filename=PersonalMessages.csv");
return File(new DownloadService().GetPersonalMessages(), "text/csv");

score 22 · Answer 1 · edited May 23 '17 at 12:09

22

Zareth's answer helped the OP, but it didn't actually answer the question. Here's the correct solution, from this other post:

public ActionResult Download()
{
    var data = Encoding.UTF8.GetBytes("some data");
    var result = Encoding.UTF8.GetPreamble().Concat(data).ToArray();
    return File(result, "application/csv", "foo.csv");
}

The byte-order mark (while not technically required for UTF8) clues certain programs (e.g. Excel >2007) in to the fact that you're using UTF8. You have to manually include it via the GetPreamble() method.

edited May 23 '17 at 12:09

Community

1
1

answered Jul 09 '12 at 16:32

StriplingWarrior

151,543
27
246
315

The MIME type should be: `text/csv`, [see here](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types) (and if you want to be more precise then use: `text/csv; charset=utf-8`, [see here](https://www.w3.org/International/articles/http-charset/index#charset)). – Ofir Mar 13 '18 at 09:55
@fini007: I think it depends on the intent. [According to this SO post](https://stackoverflow.com/a/18927941/120955), "If a server says 'This data is of type text/csv' the client can understand that can render that data internally, while if the server says 'This data is of type application/csv' the client knows that it needs to launch the application that is registered on the OS to open csv files." If you want the browser to launch Excel, then "application/csv" is probably better. Of course, if you're using content-disposition to make the file download then that's kind of moot. – StriplingWarrior Mar 13 '18 at 18:02

score 4 · Accepted Answer · answered Nov 26 '10 at 08:26

4

You might want to try using the UTF8Encoding class. The constructor has a parameter that determines if it should provide the BOM or not. You'll probably have to use the GetBytes-method and write the string as a series of bytes in the response, and not convert it back into a .net string object.

answered Nov 26 '10 at 08:26

Zareth

477
2
5

1

Thanks, changed my code to System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding(true); but still no luck... – Michel Nov 26 '10 at 08:40
1

you did set me on the right track by the getbytes and all. I was on the wrong track because notepad++ said it was a utf8 not bom file, and when i converted it to utf8, it worked. But of course utf8 was not a requirement on my side. I've tried it with a unicodeencoding and tadaa! it worked. Thanks for your thinking. – Michel Nov 26 '10 at 08:51
1

FYI: For some reason the constructor parameter doesn't actually cause it to output the BOM. You have to prepend the result of `.GetPreamble()` manually. – StriplingWarrior Jul 10 '12 at 16:27

score 1 · Answer 3 · answered Nov 26 '10 at 08:19

1

You could simplify your code a little:

public ActionResult DownloadPersonalMessages()
{
    StringBuilder myCsv = new StringBuilder();
    myCsv.Append(new DownloadService().GetPersonalMessages());
    Response.AddHeader("Content-Disposition", "attachment; filename=PersonalMessages.csv");
    return File(Encoding.UTF8.GetBytes(myCsv.ToString()), "text/csv");
}

As far as the UTF-8 encoding is concerned I am afraid the problem might be in this GetPersonalMessages method. You might need to return a stream or byte array which you could directly return as file.

answered Nov 26 '10 at 08:19

Darin Dimitrov

1,023,142
271
3,287
2,928

will try that. And the code became so large by trying all different things :) – Michel Nov 26 '10 at 08:23
hmm, when i return a byte array, will that make it UTF8? Wll, i'll give it a try. The GetPersonalMessgages() function reads a nvarchar field from the database btw – Michel Nov 26 '10 at 08:26

How to make sure my created filedownload is UTF-8? (and not UTF-8 without BOM)

3 Answers3

Linked