Advanced TextReader to EndOfFile

Question

I have a textReader that in a specific instance I want to be able to advance to the end of file quickly so other classes that might hold a reference to this object will not be able to call tr.ReadLine() without getting a null.

This is a large file. I cannot use TextReader.ReadToEnd() as it will often lead to an OutOfMemoryException

I thought I would ask the community if there was a way SEEK the stream without using TextReader.ReadToEnd() which returns a string of all data in the file.

Current method, inefficient.
The following example code is a mock up. Obviously I am not opening a file with an if statement directly following it asking if I want to read to the end.

TextReader tr = new StreamReader("Largefile");
if(needToAdvanceToEndOfFile)
{
    while(tr.ReadLine() != null) { }
}

Desired solution (Note this code block contains fake 'concept' methods or methods that cannot be used due to risk of outofmemoryexception)

TextReader tr = new StreamReader("Largefile");
if(needToAdvanceToEndOfFile)
{
    tr.SeekToEnd(); // A method that does not return anything. This method does not exist.
    // tr.ReadToEnd() not acceptable as it can lead to OutOfMemoryException error as it is very large file.
}

A possible alternative is to read through the file in bigger chunks using tr.ReadBlock(args).

I poked around ((StreamReader)tr).BaseStream but could not find anything that worked.

As I am new to the community I figured I would see if someone knew the answer off the top of their head.

Here: http://stackoverflow.com/questions/2161895/reading-large-text-files-with-streams-in-c — Random, Jul 26 '11 at 02:18
If I am reading the link you posted correctly, the only item I can see that might apply is the tr.Read() / tr.ReadBlock() which would allow me to read through the file in bigger chunks. I did mention that as a possibility above. I'm hoping someone knows a way to advance the stream to the end using ((StreamReader)tr).BaseStream or another method that does not use loops. Thanks! — JHubbard80, Jul 26 '11 at 02:27

Jalal Said · Answer 1 · 2011-07-26T02:39:30.100

2

Use

reader.BaseStream.Seek(0, SeekOrigin.End);

Test:

using (StreamReader reader = new StreamReader(@"Your Large File"))
{
    reader.BaseStream.Seek(0, SeekOrigin.End);

    int read = reader.Read();//read will be -1 since you are at the end of the stream
}

Edit: Test it with your code:

using (TextReader tr = new StreamReader("C:\\test.txt"))//test.txt is a file that has data and lines
{
    ((StreamReader)tr).BaseStream.Seek(0, SeekOrigin.End);

    string foo = tr.ReadLine();
    Debug.WriteLine(foo ?? "foo is null");//foo is null
    int read = tr.Read();
    Debug.WriteLine(read);//-1
}

edited Jul 26 '11 at 02:39

answered Jul 26 '11 at 02:20

Jalal Said

15,906
7
45
68

Already did. ((StreamReader)tr).BaseStream.Seek(0, SeekOrigin.End); does not work. tr.ReadLine() returns string data still. – JHubbard80 Jul 26 '11 at 02:26
1

I use `StreamReader` and test it with a file, the `read` will have `-1` **Edit:** I even test it will ` ((StreamReader)tr).BaseStream.Seek(0, SeekOrigin.End);` and the `read` after that returns `-1` – Jalal Said Jul 26 '11 at 02:28
I just did this test (code snippet) if ((line = tr.ReadLine()) != null) { if (line.StartsWith("#%GroupName=") || line.StartsWith("#%SetName=")) { ((StreamReader)tr).BaseStream.Seek(0, SeekOrigin.End); string s = tr.ReadLine(); // returned #%Columns=5 – JHubbard80 Jul 26 '11 at 02:33
I test it and it returns `null`. – Jalal Said Jul 26 '11 at 02:33
I do not know what to say. I do it on my end using the code above and tr.ReadLine() still returns a string. – JHubbard80 Jul 26 '11 at 02:38
I test it with your code and the results is as expected. "check the answer update" – Jalal Said Jul 26 '11 at 02:41
1

Your method only works if the file stream is at the beginning. That is why it was not working. I am in the middle of the file when I need to seek to the end. I tried ((StreamReader)tr).BaseStream.Seek(((StreamReader)tr).BaseStream.Position, SeekOrigin.End) in case the first argument was asking for the current offset position in the file, and that did not work either. – JHubbard80 Jul 26 '11 at 02:44
Yes you are right I tested it, I will try to think in way around it – Jalal Said Jul 26 '11 at 02:51
The `StreamReader` class buffers part of the file. Unless you want to write your own StreamReader class, you'll have this problem. – Hand-E-Food Jul 26 '11 at 03:06

score 2 · Accepted Answer · answered Jul 26 '11 at 02:50

You have to discard any buffered data if you have read any file content - since data is buffered you might get content even if you seek the underlying stream to the end - working example:

StreamReader sr = new StreamReader(fileName);
string sampleLine = sr.ReadLine();

//discard all buffered data and seek to end
sr.DiscardBufferedData();
sr.BaseStream.Seek(0, SeekOrigin.End);

The problem as mentioned in the documentation is

The StreamReader class buffers input from the underlying stream when you call one of the Read methods. If you manipulate the position of the underlying stream after reading data into the buffer, the position of the underlying stream might not match the position of the internal buffer. To reset the internal buffer, call the DiscardBufferedData method

This worked great! Thank both you and Jalal Aldeen Saa'd for your assistance! — JHubbard80, Jul 26 '11 at 02:55

Advanced TextReader to EndOfFile

2 Answers2