62

The MSDN site states:

A buffer is a block of bytes in memory used to cache data, thereby reducing the number of calls to the operating system. Buffers improve read and write performance. A buffer can be used for either reading or writing, but never both simultaneously. The Read and Write methods of BufferedStream automatically maintain the buffer.

Should I use this class in every possible occasion?

Jader Dias
  • 88,211
  • 155
  • 421
  • 625

6 Answers6

74

According to Brad Abrams, almost never: link

No, there is zero benefit from wrapping a BufferedStream around a FileStream. We copied BufferedStream’s buffering logic into FileStream about 4 years ago to encourage better default performance... In fact, I don’t think there are any Streams in the .NET Framework that require it, but it might be needed by custom Stream implementations if they do not do buffering by default.

Mike Chamberlain
  • 39,692
  • 27
  • 110
  • 158
dewald
  • 5,133
  • 7
  • 38
  • 42
  • 19
    -1 `FileStream` is but one of the .net streams, and writing e.g. single bytes to `NetworkStream`, which does not override `Stream`'s generic `WriteByte()` method, is highly inefficient. `BufferedStream` will boost the performance of this and other use cases tremendously (see @idn 's answer) – Evgeniy Berezovsky Aug 21 '14 at 01:03
  • Note that even with a `FileStream` I've seen benefits buffering downstream if there is an operation after the `FileStream`, eg. `FileStream` -> `Decompression` -> `BufferedStream` where the `Decompression` step doesn't buffer internally. – playsted Apr 08 '20 at 14:32
  • Ironically, if it's true that the "buffering logic" of `BufferedStream` was "copied into [most] Streams in the .NET Framework...", then doing so *froze* that logic by hard-coding it into each of them, and the remaining use for `BufferedStream` today is actually in ***replacing,*** rather than ***wrapping*** those concrete `Stream` implementations, in order to adjust, tweak, or repair any flaws in that "logic". Most notably, the internal **`Flush`** behavior of the concrete streams cannot be changed, and may not be desirable in [certain scenarios](https://stackoverflow.com/a/57837393/147511). – Glenn Slayden Feb 16 '22 at 22:21
  • I think there is some new info on this, as of .net core 3.1 - 6, it seems that adding a `BufferedSteam` does in certain cases have a positive impact on performance. This seems to be a bug that was introduced, but nonetheless when using `GZipStream` it looks like it's valid. Source: https://github.com/dotnet/runtime/issues/39233#issuecomment-745572680 – leon.io Oct 19 '22 at 00:56
22

Best case I know of is when BinaryFormatter serialize/deserialize directly from NetworkStream. Use of BufferedStream inbetween increase performance tenfold.

idn
  • 241
  • 2
  • 2
  • You're right, I used it to buffer out TcpClient's built-in stream to buffer out binary serialization directly into a stream which resulted in an enormous amount of 4 byte packets being sent out. – ricebus Jun 11 '17 at 07:21
21

The following is some text from an online course I am taking:

The BufferedStream class is a concrete class that extends the Stream class and is used to provide an additional memory buffer to another type of stream, both synchronously and asynchronously. The BufferedStream class must be configured to either read or write when an instance of the class is created, but the BufferedStream cannot be configured to perform both the tasks at the same time.

Microsoft improved the performance of all streams in the .NET Framework by including a built-in buffer. The performance noticeably improved by applying a BufferedStream to existing streams, such as a FileStream or MemoryStream. Applying a BufferedStream to an existing .NET Framework stream results in a double buffer.

The most common application of the BufferedStream class is in custom stream classes that do not include a built-in buffer.

hacker
  • 1,115
  • 1
  • 15
  • 28
  • 58
    I can't understand applying a buffer to a `MemoryStream`, as it is already essentially one giant `byte[]` buffer in memory. The sentence "The performance noticeably improved by applying a BufferedStream to existing streams, such as a FileStream or MemoryStream" seems incomplete, and perhaps is missing the negating words "is not". Wondering whether this is in fact a misleading answer... – Drew Noakes May 10 '10 at 05:19
  • 1
    If you just read this anwser without seeing the other answers you might be confused like myself, I believe the yellow highlighted text is contradictory, and possibly out of date, also. I think it should be ignored, other answers explain that FileStream and MemoryStream don't need BufferedStream. – Cameron Apr 30 '16 at 18:28
  • Could it be that a buffered stream on a MemoryStream will buffer part of it into a faster cache category, like L3 cache? – findusl Aug 23 '18 at 07:08
  • It is sad that I have to dig this information out on SO instead of MSDN page for `StreamReader`. – arrowd Aug 30 '21 at 18:48
  • @findusl No, apples and oranges. The general-purpose nature of the `Stream` concept itself is precisely what precludes it being able have any control over how its content is laid-out and accessed, and this is the exact and only kind of information required (in excrutiating detail) to ever even begin to effect any measurable difference due to memory bus performance. Of more realistic concern is the worst-case "memory access" of reading/writing 100% faulted/dirty VM pages (thus essentially disk I/O), but here again there's nothing a `Stream` can do to out-perform the OS page-fault mechanism. – Glenn Slayden Feb 16 '22 at 23:30
  • As mentioned above, there is some updated info from the .net team members which shows that for `GZipStream` a ` BufferedWriter` should be considered. Source: https://github.com/dotnet/runtime/issues/39233#issuecomment-745572680 – leon.io Oct 19 '22 at 00:58
6

Late answer but going to answer it anyways. If you understand how it works you can understand why it makes no sense to use in some cases and why it makes sense to not use in other cases.

To proof this I am using StreamWrapper class. Only use this class to see how many times you hit a breakpoint! Place breakpoints all over this class. Our goal is to see how many times we call the Write, Read, and other methods.

// This class is only used for demo purposes. Place a breakpoint on all parts
class StreamWrapper : Stream
{
    Stream stream;

    public StreamWrapper(Stream s)
    {
        stream = s;
    }

    public override bool CanRead => stream.CanRead;

    public override bool CanSeek => stream.CanSeek;

    public override bool CanWrite => stream.CanWrite;

    public override long Length => stream.Length;

    public override long Position { get => stream.Position; set => stream.Position = value; }

    public override void Flush()
    {
        stream.Flush();
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        return stream.Read(buffer, offset, count);
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        return stream.Seek(offset,origin);
    }

    public override void SetLength(long value)
    {
        stream.SetLength(value);
    }

    public override void Write(byte[] buffer, int offset, int count)
    {
        stream.Write(buffer, offset, count);
    }


}

Now that you have that wrapper class that is basically a wrapper to an existing strem you may do the following tests:

Example Writting:

// in real life you want this to be larger. 
int bufferSize = 8;


// Use BufferedStream to buffer writes to a MemoryStream.
using (var memory_test = new StreamWrapper(new MemoryStream()))
using (BufferedStream stream = new BufferedStream(memory_test, bufferSize))
{


    // all this will only send one write to memory_test!
    stream.Write(new byte[] { 1, 2 });
    stream.Write(new byte[] { 1, 2 });
    stream.Write(new byte[] { 1, 2 });
    stream.Write(new byte[] { 1, 2 });
    // BREAKPOINT ONLY HITS ONE TIME 


    // All this will also send only one write to memory_test
    for (int i = 0; i < 8; i++)                            
        stream.WriteByte(5);
    // BREAKPOINT ONLY HITS ONE TIME AGAIN INSTAD OF 8


    // this will send one write to memory_test. Writes of more than 8 bytes can happen!
    stream.Write(new byte[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18 });
    // ALL THIS WILL SEND ONE WRITE AGAIN
}

Example reading:

// example reading
{
    // create stream with some data in it that we will be reading
    var ms = new MemoryStream();
    {
        // Write 256 bytes
        for (int i = 0; i <= byte.MaxValue; i++)
        {
            ms.WriteByte((byte)i);
        }
        ms.Position = 0;
    }

    // Use BufferedStream to buffer writes to a MemoryStream.
    using (var memory_test = new StreamWrapper(ms))
    {                                                        
        using (BufferedStream stream = new BufferedStream(memory_test, 8))
        {
            // note I do not care about the output of each read for demo breakpoint. On real life you will store that output
            // for now we only care how many times we hit the breakpoint because on real life that could be a slow/expensive 
            // operation such as opening a file for writing and you want to do those as few as possible.

            // hit breakpoint only one time with all this reads
            stream.ReadByte();
            stream.ReadByte();
            stream.ReadByte();
            stream.ReadByte();
            stream.ReadByte();
            stream.ReadByte();
            stream.ReadByte();
            stream.ReadByte();


            // hit breakpoint only one time with all this reads
            stream.Read(new byte[2] );
            stream.Read(new byte[2] );
            stream.Read(new byte[2] );
            stream.Read(new byte[2] );


            // hit breakpoint only one time even though it is larger than our buffer size 8
            // our goal is to hit the breakpoint as fewest time as possible because in real life
            // this could be slow/expensive operations
            stream.Read(new byte[1024] );

        }
    }
}

If this where to be write operations to a file for example performance could increase by ensuring you always write at least 8 bytes instead of 1 at a time. In real life you want this number to be about 4 KB

Tono Nam
  • 34,064
  • 78
  • 298
  • 470
5

The normal file I/O streams are already buffered by using a StreamReader/StreamWriter.

Since read/write operations on streams, normally use the Read/Write methods that take a byte array, you will naturally provide some buffering yourself.

If you use very small arrays, or use WriteByte, you might get better performance by using a BufferedStream in between.

GvS
  • 52,015
  • 16
  • 101
  • 139
3

What must be used in every possible occasion is common sense. There's no use in utilizing this class when reading-writing to-from a MemoryStream, but it might be quite useful when doing network or disk IO (if Streams for these subsystems do not do buffering on their own).

Anton Gogolev
  • 113,561
  • 39
  • 200
  • 288
  • Do you have a definitive list that shows which streams are buffered and which aren't? I hit this question, specifically interesting in `NetworkStream`. I'd like to lose some buffering code as I'm pretty sure it's no needed. Will keep browsing! – Drew Noakes May 10 '10 at 05:22
  • Turns out, for the `NetworkStream` case, that you can only read and write using `byte[]`, so the buffering is controlled by the caller. `NetworkStream` is also not seekable, so no internal buffer is required. – Drew Noakes May 10 '10 at 05:47