Copying from one stream to another?

Question

For work, the specification on my project is to use .Net 2.0 so I don't get the handy CopyTo function brought about later on.

I need to copy the response stream from an HttpWebResponse to another stream (most likely a MemoryStream, but it could be any subclass of Stream). My normal tactic has been something along the lines of:

BufferedStream bufferedresponse = new BufferedStream(HttpResponse.GetResponseStream());
int count = 0;
byte[] buffer = new byte[1024];
do {
    count = bufferedresponse.Read(buffer, 0, buffer.Length);
    target.Write(buffer, 0, count);
} while (count > 0);
bufferedresponse.Close();

Are there more efficient ways to do this? Does the size of the buffer really matter? What is the best way to copy from one stream to another in .Net 2.0?

P.S. This is for downloading large, 200+ MB GIS tif images. Of course reliability is paramount.

Possible duplicate of [Best way to copy between two Stream instances](http://stackoverflow.com/questions/230128/best-way-to-copy-between-two-stream-instances-c). Also the chosen answer is what you're currently doing, so I'd say you're doing it correctly. — Chris Walsh, Jun 02 '11 at 15:09
Before I get my question closed, I would like to say that mine comes with the .Net 2.0 restriction. I hope that's enough to keep it open, I don't want my questions closed :( — Corey Ogburn, Jun 02 '11 at 15:20
The question I linked to has an answer using your method as well as the newer `CopyTo` method. It's worth mentioning that you would get better performance (if it fits your use case) to wait for the whole stream to come in and then copy the stream over in larger chunks. But that's not always ideal. — Chris Walsh, Jun 02 '11 at 15:22

score 9 · Accepted Answer · answered Jun 02 '11 at 15:14

This is a handy function. And yes, the buffer size matters. Increasing it might give you better performance on large files.

public static void WriteTo(Stream sourceStream, Stream targetStream)
{
       byte[] buffer = new byte[0x10000];
       int n;
       while ((n = sourceStream.Read(buffer, 0, buffer.Length)) != 0)
           targetStream.Write(buffer, 0, n);
}

score 3 · Answer 2 · answered Jun 02 '11 at 15:18

The size of the buffer does matter. If you're copying a megabyte of data one byte at a time, for example, you're going to make a 2^20 iterations through the loop. If you're copying 1 kilobyte at a time, you'll only make 2^10 iterations through the loop. There is significant overhead in the calls to Read and Write when you're making a million of them.

For reading FileStream, I typically use a buffer that's between 64K and 256K. Anything less than 32K shows a marked decrease in performance, as does anything above 256K. The difference between using a 64K buffer and a 256K buffer is not worth the extra memory. Be aware, though, that those numbers are on my system and across my network. Your numbers will vary depending on hardware and operating system.

For network streams, you should select a buffer size that will keep up with the incoming data stream. I'd suggest at least 4 kilobytes, which will give you some buffer if the write stalls for any reason.

Guffa · Answer 3 · 2011-06-02T15:30:32.970

You can get rid of the BufferedStream, it's only useful if you are reading small chunks of the stream. Just get the response stream in a variable and use that:

Stream response = HttpResponse.GetResponseStream();

A buffer that is too small will reduce the performance. You can use a larger buffer, so that at least the data from an entire IP packet fits. I looked around a bit, and 4096 bytes should be enough for that. You can really use any size up to 85 kb, after that it's allocated in the large objects heap, which you should avoid to do when there is no reason for it.

Other than that, it's about as efficient as it gets.

score 0 · Answer 4 · answered Jun 02 '11 at 15:20

0

I can think of two ways.

Check if Stream.MemberWiseClone() fits your need. It gets you a shallow copy of your object.
Check if this works, if both the ends are of the Stream type.

BufferedStream bs = new BufferedStream((Stream)memoryStreamObject);

answered Jun 02 '11 at 15:20

SaravananArumugam

3,680
6
33
45

Those methods will just give you another way of accessing the same stream, for that purpose you can just copy the stream reference to another variable... – Guffa Jun 02 '11 at 15:34

Copying from one stream to another?

4 Answers4

Linked