How to read a very huge file from the end in C#?

Question

I have a text file of size 4.5GB, which was created by one of the experimental applications. On my system of 8GB RAM, this file cannot be opened (tried with sublime text). I just want to read the content of this file from the end, say 45 characters from the end, without loading the entire file in memory. How to do this ? The inbuilt methods like,

StreamReader.ReadBlock(char[] buffer,int index,int count)

has type int for index, but the index for my case is out of the range for int value. If only there was an overload with long. How to overcome this ? Thanks.

[`FileStream.Seek(long offset, SeekOrigin origin)`](http://msdn.microsoft.com/en-us/library/system.io.filestream.seek(v=vs.110).aspx)? Darin Dimitrov shows an example how to do this: http://stackoverflow.com/questions/4368857/read-from-a-file-starting-at-the-end-similar-to-tail — wkl, Aug 29 '14 at 19:37
What encoding is the text file in? If it is UTF-8 can you guarantee that there are only single byte wide characters in the file? — Scott Chamberlain, Aug 29 '14 at 19:40
@ScottChamberlain YES, since the output is from my own application, its guaranteed that the file is made up of ASCII characters only. — MrClan, Aug 29 '14 at 19:43
@birryree Thanks, that is exactly what I was looking for. SO didn't suggest that link while I was typing that question. :) — MrClan, Aug 29 '14 at 19:46

Lucas Trzesniewski · Accepted Answer · 2014-08-29T19:50:24.463

A Stream has long properties named Position and Length. You can set the Position to any value you like within the range [0, Length[, like fileStream.Position = fileStream.Length - 45;.

Or, you could use the Stream.Seek function: fileStream.Seek(-45, SeekOrigin.End).

Use the more readable one depending on the surrounding code and situation, the one that best conveys your intention.

Here's some code:

using (var fileStream = File.OpenRead("myfile.txt"))
using (var streamReader = new StreamReader(fileStream, Encoding.ASCII))
{
    if (fileStream.Length >= 45)
        fileStream.Seek(-45, SeekOrigin.End);

    var value = streamReader.ReadToEnd(); // Last 45 chars;
}

This is easier since you know your file is encoded in ASCII. Otherwise, you'd have to read a bit of text from the start of the file with the reader to let it detect the encoding (if a BOM is present), and only then seek to the position you want to read from.

The problem is `StreamReader` is not derived from `Stream`. You should show how the two interact. — Scott Chamberlain, Aug 29 '14 at 19:45
@Scott, I've assumed the OP already had a file stream, but you're right, some example code is appropriate here. — Lucas Trzesniewski, Aug 29 '14 at 19:52

How to read a very huge file from the end in C#?

1 Answers1