StreamReader does not read the last character

Question

MyFile.txt contains (in Hex):

40 D8 40

I print all of its characters by:

using (StreamReader sr = new StreamReader("MyFile.txt"))
{
    while (!sr.EndOfStream)
    {
         int n = sr.Read();
         Console.WriteLine("{0:X}", n);
    }
}

The output is:

40
FFFD
40

But, when MyFile.txt contains the same chars while D8 is the last one:

40 40 D8

The output is only:

40
40

Where is the last char D8 (FFFD)??

Garbage in, garbage out. The Encoding that it uses needs to match the file content, you are not providing properly encoded utf8. If you are only interesting in the binary content then you need to use FileStream. — Hans Passant, Dec 12 '17 at 14:54
add this line to check for nulls prior inside your while loop `if ((n= sr.ReadLine()) != null)` — MethodMan, Dec 12 '17 at 14:56
@STLDeveloper It's never mind, this is the content from hex-editor... — Roi Bar, Dec 12 '17 at 15:11
@dlatikay 0xD8 is _nothing_ in ASCII. ASCII is 7-bit and only goes up to 0x7F. — Nyerguds, Feb 02 '18 at 23:57
right. my answer made this stale comment obsolete, I deleted it — Cee McSharpface, Feb 03 '18 at 00:04

Cee McSharpface · Answer 1 · 2017-12-15T15:20:45.667

StreamReader is expecting the input to be utf8. Unless you specify a different encoding, or use a binary FileStream instead, you will get invalid code points represented as 0xFF sequences in the output.

In your actual case, 0xD8 is a valid Unicode prefix aka "plane selector", but 0x40 is not a valid continuation byte to form a codepoint with: http://www.fileformat.info/info/unicode/char/d840/index.htm

U+D840 is not a valid unicode character.

Related, interesting read - in your second situation, the last char is missing because a prefix alone does not resolve into a character. The framework expects a continuation byte to determine a valid codepoint, but encounters end-of-stream and outputs nothing.

StreamReader does not read the last character

1 Answers1