3

I want to make a multi threaded downloader, and each thread needs to have access to the output stream and needs a independent stream object with a independent stream position.

With filestreams i can easily archieve this through FileShare.ReadWrite and opening a independent stream instance for each thread but how can i archieve the same with normal streams? This is how the multitheaded download works using multiple filestreams:

// calculate the download chunk size
var loops = (int)Math.Ceiling((double)total / chunksize);
int[] nums = new int[loops];
for (int i = 0; i < loops; i++) nums[i] = i;

// run 8 download threads in parralel
// each thread downloads a chunk to the same file
await Util.ParallelForEachAsync(nums, (chunkIndex) =>
{
    var currentPos = (chunkIndex * chunksize);
    HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Get, url);
    message.Headers.Range = new System.Net.Http.Headers.RangeHeaderValue(currentPos, currentPos + chunksize - 1);

    // each thread is able to get a independent filestream object. How can i archieve this with normal streams?
    using (FileStream f = File.Open(filename, FileMode.OpenOrCreate, FileAccess.Write, FileShare.Write))
    {
        // write message content to stream
        await ContentToStreamAsync(f, message, onProgress);
    }
}, 8);

With this downloading method i am able to archive incredible downloadspeeds compared to a single theaded download.

My question is how i can archieve the same with a stream (instead of filestream)? This is difficult because normal streams dont support a ShareMode and therefore cant be shared with multiple threads.

SinOfficial
  • 536
  • 5
  • 15
  • 1
    Have you looked at [Stream.Synchronized](https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.synchronized?view=netframework-4.8)? – Pavel Anikhouski Jan 01 '20 at 19:18
  • 1
    @PavelAnikhouski Doesnt the resulting stream use the same stream position? If all the theads write at different locations in the stream, would it still work? – SinOfficial Jan 01 '20 at 19:30
  • 1
    I think it prevents an access from a different threads. For your case you should write something your own, I suppose, like in this [thread](https://stackoverflow.com/questions/3721552/implementing-async-stream-for-producer-cosumer-in-c-sharp-net) – Pavel Anikhouski Jan 01 '20 at 19:35

0 Answers0