IO Reads for the File C#

Question

I am reading all files(around 3000 files and size is 50 GB) from specified path with 4k bytes at a time. Below is the code for the same. My query is when i see the CPU and Memory of the application in task manager i could see that the IO Reads are gradually increasing to high level, i can understand that it might be because of 4k read but does that affect to other things or its ok to increase the IO Read. Also is FileStream the optimum way to read the file as it does not load the entire file in memory?

fileStream = new FileStream(filePath, FileMode.Open, FileAccess.Read)

do
{
   BytesRead = fileStream.Read(Buffer, 0, MAX_BUFFER);
}
while (BytesRead != 0);

fileStream.Close();

I'm confused as to what the question is here. You can't read 50GB of data without doing a lot of IO... what are you trying to ask? — Jon Skeet, Apr 16 '13 at 07:39

score 1 · Accepted Answer · edited May 23 '17 at 12:04

Check Hans Passant's answer about this issue, i find it very clear.

Files are already buffered by the file system cache, You just need to pick a buffer size that doesn't force FileStream to make the native Windows ReadFile() API call to fill the buffer too often. Don't go below a kilobyte, more than 16 KB is a waste of memory.

Take a look at this post too, it provides some benchmarking code.

IO Reads for the File C#

1 Answers1