C# IInputStream missing segmented TCP package

Question

I am writing a Small HttpServer, sometime I encounter a problem with missing POST Data.

By using Wireshark I discovered, that the Header is split into two segments.

I only get the first segment (636 Bytes), the second one (POST Data in this case) gets totally lost.

Here is a the relevant C# Code

string requestHeaderString = "";
StreamSocket socketStream = args.Socket;

IInputStream inputStream = socketStream.InputStream;
byte[] data = new byte[BufferSize];
IBuffer buffer = data.AsBuffer();

try
{
    await inputStream.ReadAsync(buffer, BufferSize, InputStreamOptions.Partial);

    // This is where things go missing, buffer.ToArray() should be 678 Bytes long, 
    // so Segment 1 (636 Bytes) and Segment 2 (42 Bytes) combined.
    // But is only 636 Bytes long, so just the first Segment?!
    requestHeaderString += Encoding.UTF8.GetString(buffer.ToArray());
}
catch (Exception e)
{
    Debug.WriteLine("inputStream is not readable" + e.StackTrace);
    return;
}

This code is in part of the StreamSocketListener ConnectionReceived Event.

Do I manually have to reassemble the TCP Segments, isn't this what the Systems TCP Stack should do?

Thanks, David

You're ignoring the integer value returned by `recv()` (or its wrapper `Read()`), which is about the first issue people encounter while writing their own code using sockets. Have you tried searching? After you fixed that issue, you may want to read http://stackoverflow.com/questions/27228343/what-are-the-consequences-of-not-including-a-content-length-header-in-a-server-r — CodeCaster, Jan 11 '17 at 18:46
_Do I manually have to reassemble the TCP Segments, isn't this what the Systems TCP Stack should do?_ Yes, you do have to reassemble the parts. TCP simply guarantees that the **order** of the parts will be correct. It makes no promises about how that data will arrive; it could be broken up into smaller pieces, or multiple "sends" could arrive chunked together. It's up to you, the programmer to know when a complete "message" has arrive and extract that from your own buffer. You're also ignoring how many bytes were returned, which means the call to GetString() might return garbage. — Idle_Mind, Jan 11 '17 at 18:48
A number of problems. Principally, you're ignoring the return value, which is where you should actually get your data from. The [documentation](https://msdn.microsoft.com/library/windows/apps/br241719) couldn't be clearer about this. *"Always read data from the buffer returned in the IAsyncOperationWithProgress(IBuffer, UInt32). Don't assume that the input buffer contains the data. Depending on the implementation, the data that's read might be placed into the input buffer, or it might be returned in a different buffer."* You should try reading it. — spender, Jan 11 '17 at 18:52
In general, you request a number of bytes with a read operation, but most stream APIs are not bound to return you that number of bytes. This is why you have to check the return values from the read operation to find out exactly how much data you got back. — spender, Jan 11 '17 at 18:55

Scott Chamberlain · Accepted Answer · 2017-01-11T18:51:52.420

The problem is the systems TCP stack treats the TCP stream just like any other stream. You don't get "messages" with streams, you just get a stream of bytes.

The receiving side has no way to tell when one "message" ends and where the next begins without you telling it some how. You must implement message framing on top of TCP, then on your receiving side you must repeatedly call Receive till you have received enough bytes to form a full message (this will involve using the int returned from the receive call to see how many bytes where processed).

Important note: If you don't know how many bytes you are expecting to get in total, for example you are doing message framing by using '\0' to seperate messages you may get the end of one message and the start of the next in a single Receive call. You will need to handle that situation.

EDIT: Sorry, I skipped over the fact you where reading HTTP. You must follow the protocol of HTTP. You must read in data till you see the pattern \r\n\r\n, once you get that you must parse the header and decode how much data is in the content portion of the HTTP message then repeatatly call read till you have read the number of bytes needed.

Yeah, HTTP's message framing is a *huge* pain to get right. I recommend using ASP.NET Core, or [WebListener](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/servers/weblistener) if [one of the fastest stacks](https://www.ageofascent.com/2016/02/18/asp-net-core-exeeds-1-15-million-requests-12-6-gbps/) isn't fast enough... — Stephen Cleary, Jan 11 '17 at 19:12

C# IInputStream missing segmented TCP package

1 Answers1