HttpClient and PushStreamContent

Question

I use PushStreamContent with my REST API (ASP.NET Web API) and works great. The HttpClient can request a ressource and gets the HTTP-Response before the complete request is handled by the server (the server still writes to the push-stream).

As HttpClient you have to do one little thing: Use HttpCompletionOption.ResponseHeadersRead.

Now my question: Is it possible to to this the other way? From the HttpClient -> uploading data via a push-stream to the web api?

I Implemented it as below, but the web api gets the request not before the client closes the stream.

         var asyncStream = new AsyncStream(fs);
         PushStreamContent streamContent = new PushStreamContent(asyncStream.WriteToStream);
         content.Add(streamContent);

         HttpResponseMessage response = await c.SendAsync(new HttpRequestMessage(new HttpMethod("POST"), "http://localhost/...") { Content = content }, HttpCompletionOption.ResponseHeadersRead);

         response.EnsureSuccessStatusCode();

The AsyncStream is my class with the delegate:

public async void WriteToStream(Stream outputStream, HttpContent content, TransportContext context)

This is necessary for the Push-Stream.

Is this possible somehow? The HttpClient do not send the request to the web api until the last bytes are written to the stream...

What do I have to do? Is the problem on the client side or maybe on the server / asp.net web api-side?

Edit: This is the implemenation of WriteToStream (but I do not use a file from disk, is use a memorystream 'myMemoryStream' (passed in the constructor):

public void WriteToStream(Stream outputStream, HttpContent content, TransportContext context)
{
    try
    {
        var buffer = new byte[4096];

        using (var stream = myMemoryStream)
        {
            var bytesRead = 1;

            while (bytesRead > 0)
            {
                bytesRead = video.Read(buffer, 0, buffer.Length);
                outputStream.Write(buffer, 0, bytesRead);
            }
        }
    }
    catch (HttpException ex)
    {
        return;
    }
    finally
    {
        outputStream.Close();
    }
}

Maybe I have to do something with: HttpContent content, TransportContext context ?

score 5 · Accepted Answer · edited May 23 '17 at 12:09

I found the solution to my problem:

I want to set: httpWebRequest.AllowReadStreamBuffering = false;

HttpClient 4.0 does buffering by default and you cannot acces the property AllowReadStreamBuffering, so you have to use HttpWebRequest directly. (Or you can use HttpClinet 4.5, there is the default behaviour 'streaming') see: http://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/ 6. Using HttpClient)

The second problem was fiddler: Fiddler currently only supports streaming of responses and not requests (Fiddler makes HttpWebRequest/HttpClient behaviour unexpected)

The solution that worked for me:

HttpWebRequest httpWebRequest = HttpWebRequest.Create(...)
httpWebRequest.Method = "POST";
         httpWebRequest.Headers["Authorization"] = "Basic " + ...;
         httpWebRequest.PreAuthenticate = true;
         httpWebRequest.AllowWriteStreamBuffering = false;
         httpWebRequest.AllowReadStreamBuffering = false;
         httpWebRequest.ContentType = "application/octet-stream";
         Stream st = httpWebRequest.GetRequestStream();
st.Write(b, 0, b.Length);
st.Write(b, 0, b.Length);
//...
         Task<WebResponse> response = httpWebRequest.GetResponseAsync();

         var x = response.Result;
         Stream resultStream = x.GetResponseStream();
//... read result-stream ...

Examples see here. My other answer with some code: http://stackoverflow.com/questions/25551887/httpwebrequest-chunked-async-post — user437899, Sep 03 '14 at 09:27
This works in *ideal* situations only and unfortunately is not a viable option. The client is stubborn enough to continue writing its whole stream, even if the server long errored and sent an HTTP status back. The client won't read anything the server sent unless it finished its request, amazingly even after shutting down the app pool, IIS, and the underlying socket got closed as well. I've tried going one level down to HttpWebRequest to attempt to read the stream while the client is still writing, however this operation is not allowed. — MoonStom, Dec 16 '15 at 04:33

score 4 · Answer 2 · answered Aug 21 '14 at 15:28

4

Make sure to do requestMessage.Headers.TransferEncodingChunked = true; on your request message...the reason is if you do not set this, HttpClient would buffer the entire content at the client itself in order to figure out the Content-length of the request and this is the reason you are noticing your web api service not being invoked immediately as you are writing to the stream...

answered Aug 21 '14 at 15:28

Kiran

56,921
15
176
161

Hi, thank you this is a good information. Unfortunately it does not work. The client still waits for the close() of the stream and only then he sends the request. The other way: If the server writes the first bytes to the stream I get a reaction on the client side. But if the client writes the first bytes, nothing happens. It writes to the end and closes the stream. Then the request is send. Strangely if the server reads at the stream he gets 0 bytes... (This only happens if I activate Chunking on the client-httprequestmessage) – user437899 Aug 22 '14 at 14:34
if your service is a ASP.NET Web API service hosted in IIS, then the default settings for incoming requests is to buffer the body and so you need to make the a change to the `IHostBufferPolicySelector` service to make the request body to be not buffered...example: http://stackoverflow.com/questions/12699364/implementing-system-web-http-webhost-webhostbufferpolicyselector-ihostbufferpoli .. – Kiran Aug 22 '14 at 14:43
I already use IHostBufferPolicySelector, but this is not my problem. I checked the call of the interface's method, but it is not called until the client sends the request. I have the web api hosted as service as well (self hosted). There I noticed that my web api get the stream - even if the client uses TransferEncodingChunked = true. (web hosted the web api gets a stream with no content). Also I have noticed a different to web hosted. The stream I get is not seekable, so i have to change my method, which used the length-property. But still: Chunking does not work. The client have to write... – user437899 Aug 22 '14 at 16:52
... the whole date to the stream and have to call stream.close() and then the HttpClient sends the request. So I think there is a problem on the client side. Something I have to configure or else... Basically, is it possible that http-clients can POST requests and send the data via streams after (or while) the server is still processing the request (and is reading on the stream). Does the HttpClient supports 'PushStreamContent'? I had to add a additional reference. Until then the HttpClient did not know the 'PushStreamContent'. – user437899 Aug 22 '14 at 16:57
can you share your implementation of WriteToStream? – Kiran Aug 22 '14 at 17:11
I got the implementation from: http://stackoverflow.com/questions/16168683/webapi-streamcontent-vs-pushstreamcontent I will insert my implementation to the question – user437899 Aug 22 '14 at 17:57
I am confused...if you are using filestream or a memorystream why are you using PushStreamContent to begin with...you can just use StreamContent instead – Kiran Aug 22 '14 at 18:29
This is only exemplary. Later the PushStreamContent should dynamically write the content. – user437899 Aug 23 '14 at 15:29

HttpClient and PushStreamContent

2 Answers2

Linked