Irregular character/text encoding issue with writing back to file

Question

I'm using this function to read text lines from a file:

string[] postFileLines = System.IO.File.ReadAllLines(pstPathTextBox.Text);

Inserting a few additional lines at strategic spots, then writing the text lines back to a file with:

TextWriter textW = new StreamWriter(filePath);
for (int i = 0; i < linesToWrite.Count; i++)
{
    textW.WriteLine(linesToWrite[i]);
}

textW.Close();

This works perfectly well until the text file I am reading in contains an international or special character. When writing back to the file, I don't get the same character - it is a box.

Ex:

Before = W:\Contrat à faire aujourdhui\ `

After = W:\Contrat � faire aujourdhui\ `

This webpage is portraying it as a question mark, but in the text file it's a rect white box.

Is there a way to include the correct encoding in my application to be able to handle these characters? Or, if not, throw a warning saying it was not able to properly write given line?

http://stackoverflow.com/questions/4520184/how-to-detect-the-character-encoding-of-a-text-file ? — spender, Nov 28 '12 at 23:18
Following up on @spender's comment http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx — nonsensickle, Nov 28 '12 at 23:18

score 2 · Accepted Answer · answered Nov 28 '12 at 23:25

2

Add encondig like this:

File.ReadAllLines(path, Encoding.UTF8);

and

new StreamWriter(filePath, Encoding.UTF8);

Hope it helps.

answered Nov 28 '12 at 23:25

jalcalav

324
3
6

thanks for the suggestion, I tried this, same result. I tried some of the other UTF encodings, however this made the entire file garbled text. I will have to probably investigate the comments and figure out how to determine the encoding or something – ikathegreat Nov 29 '12 at 00:40
UTF-8 is the right decision. Change the code specifying UTF-8, erase your actual file, run your application to generate a brand new file, open the new file using Notepad++ or other good editor (notepad is not an option because it fails reading UTF-8 witch such strange symbols). Nevertheless, if you like to use notepad in windows, then change Encoding.UTF8 for Encoding.GetEncoding(28591) (This is equivalent to iso-8859-1) – jalcalav Nov 29 '12 at 02:52

score 0 · Answer 2 · answered Sep 16 '13 at 10:53

0

use This , works for me

string txt = System.IO.File.ReadAllText(inpPath, Encoding.GetEncoding("iso-8859-1"));

answered Sep 16 '13 at 10:53

Usman Younas

1,323
15
21

score 0 · Answer 3 · edited Jul 10 '15 at 14:05

0

You can try UTF encoding while writing to the file as well,

textW.WriteLine(linesToWrite[i],Encoding.UTF8);

edited Jul 10 '15 at 14:05

Fjodr

919
13
32

answered Jul 10 '15 at 13:43

PeeduS

1

score 0 · Answer 4 · answered Jul 10 '15 at 16:08

You may be need to write Single-byte Character Sets

Using Encoding.GetEncodings() you can easily get all possible encoding. ("DOS" encoding are System.Text.SBCSCodePageEncoding)

enter image description here

In your case you may need to use

File.ReadAllLines(path, Encoding.GetEncoding("IBM850"));

and

new StreamWriter(filePath, Encoding.GetEncoding("IBM850"));

Bonne journée! ;)

Irregular character/text encoding issue with writing back to file

4 Answers4