Sending and returning a continuous large stream in one WCF operation/call

Question

I would like to send a stream of data(huge file > 2GB), to a WCF service, process it then return the processed data as a stream (transferMode = "Streamed" ), without buffering the entire stream in memory then sending it back.

I know the traditional approach of streaming data in and out(outside of a WCF operation) involves

1. At the consuming end, sending a stream to a (WCF service) void method with an input Stream and an output Stream as parameters.
2. At the consuming end, also having a receiving stream to get the incoming,processed output Stream
3. In the method, processing that input Stream then writing the processed bytes to the output Stream
4. those processed bytes are what is received through the output Stream

This way, the stream flow is not broken.

E.g from a Microsoft sample:

void CopyStream(System.IO.Stream instream, System.IO.Stream outstream)
{
    //read from the input stream in 4K chunks
    //and save to output stream
    const int bufferLen = 4096;
    byte[] buffer = new byte[bufferLen];
    int count = 0;
    while ((count = instream.Read(buffer, 0, bufferLen)) > 0)
    {
        outstream.Write(buffer, 0, count);
    }
}

I would like to do the same, only that outputstream would be a WCF return type. Is that possible? How do I do it with transferMode = "Streamed"?

Using WCF, you cannot have more than one parameter(or message contract object) of Stream type when you want to use transferMode = "Streamed"

Hypothetically, with psuedocode like this:

Stream StreamAndReturn(System.IO.Stream instream) 
{ 
    Stream outstream = new MemoryStream();//instantiate outstream - probably should be buffered?

    while ((count = instream.Read(buffer, 0, bufferLen)) > 0) 
    { 
        //some operation on instream that will 
        SomeOperation(instream,outstream);
    } 
    return outstream; //obviously this will close break the streaming
}

I also tried NetTcpBinding with SessionMode set to SessionMode.Allowed, hoping to have a session that I can start,send the stream data to the service, get the results in a separate stream then using the OperationContext, retrieve whatever property will be associated with with that service instance. But it did not retain the session information, see below:

According to MSDN documentation I should also set InstanceContextMode = InstanceContextMode.PerSession and ConcurrencyMode = ConcurrencyMode.Multiple (see last paragraph)

For that, I Asked a question on SO, but still waiting for an answer. I am thinking maybe there is a better way to do it.

Answer 1

My recommendation would be for you to create two methods:

• One that takes your input as a stream and returns an ID number generated on the server.
• Another that takes the ID number, and returns the response stream.

You'll need some way of coordinating the input and output on the server, and it will make it a little more complicated if you put this on multiple servers behind a load balancer (shared state and all that), but it will allow you to use streaming for both the request side and the response side.