FileStream.Seek vs. Buffered Reading

Question

Motivated by this answer I was wondering what's going on under the curtain if one uses lots of FileStream.Seek(-1).

For clarity I'll repost the answer:

using (var fs = File.OpenRead(filePath))
{
    fs.Seek(0, SeekOrigin.End);

    int newLines = 0;
    while (newLines < 3)
    {
        fs.Seek(-1, SeekOrigin.Current);
        newLines += fs.ReadByte() == 13 ? 1 : 0; // look for \r
        fs.Seek(-1, SeekOrigin.Current);
    }

    byte[] data = new byte[fs.Length - fs.Position];
    fs.Read(data, 0, data.Length);
}

Personally I would have read like 2048 bytes into a buffer and searched that buffer for the char.

Using Reflector I found out that internally the method is using SetFilePointer.

Is there any documentation about windows caching and reading a file backwards? Does Windows buffer "backwards" and consult the buffer when using consecutive Seek(-1) or will it read ahead starting from the current position?

It's interesting that on the one hand most people agree with Windows doing good caching, but on the other hand every answer to "reading file backwards" involves reading chunks of bytes and operating on that chunk.

@ChaosPandion: I didn't rate your comment, I was just curious. — VVS, Dec 06 '10 at 21:26

score 8 · Accepted Answer · answered Dec 06 '10 at 18:03

Going forward vs backward doesn't usually make much difference. The file data is read into the file system cache after the first read, you get a memory-to-memory copy on ReadByte(). That copy isn't sensitive to the file pointer value as long as the data is in the cache. The caching algorithm does however work from the assumption that you'd normally read sequentially. It tries to read ahead, as long as the file sectors are still on the same track. They usually are, unless the disk is heavily fragmented.

But yes, it is inefficient. You'll get hit with two pinvoke and API calls for each individual byte. There's a fair amount of overhead in that, those same two calls could also read, say, 65 kilobytes with the same amount of overhead. As usual, fix this only when you find it to be a perf bottleneck.

Ah, didn't take the cost for p/invoking into account. That's already reason enough for reading in chunks. — VVS, Dec 06 '10 at 18:12

score 1 · Answer 2 · answered Dec 06 '10 at 18:07

Here is a pointer on File Caching in Windows

The behavior may also depends on where physically resides the file (hard disk, network, etc.) as well as local configuration/optimization.

An also important source of information is the CreateFile API documentation: CreateFile Function

There is a good section named "Caching Behavior" that tells us at least how you can influence file caching, at least in the unmanaged world.

FileStream.Seek vs. Buffered Reading

2 Answers2

Linked