C# Seperates the content from the response with the header from byte array

Question

I have a byte array which looks like this if printed in a console:

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Length: 163
Content-Type: text/plain
Content-Encoding: gzip
Vary: Accept-Encoding
Server: Microsoft-IIS/8.0
Set-Cookie: ckperm=1498739450811
Set-Cookie: cknw=1498739582361
Set-Cookie: ckcd1=204510200
Set-Cookie: ckcd2=35132595069
X-Powered-By: ASP.NET
Date: Sun, 16 Jul 2017 00:27:57 GMT

      ??`I?%&/m?{J?J??t?`$??@???????iG#)?*??eVe]f@??????{????
{????;?N'????\fdl??J???!????~|?"?m?j??[????^?m6?V???l??}???????{?>?{?
3?????U-??.

The content is above that is corrupted when printed. I want to remove header and only get the content that will be return in a byte[] form. Converting the response to string and splitting the string by a newline isn't an option because it gets corrupted. How can I achieve this? Thanks!

Answer 1

To separate your HTTP request header and data:

1. Read NetworkStream And copy the data to MemoryStream
2. Call byte[] mem = MemoryStream.ToArry() Method
3. Search for these bytes { 13, 10, 13, 10 } and find first index within mem
4. And substring mem like byte[] heaader = mem[..endpoint]
5. If you want to Consume Http request body you may want to install nuget package Install-Package HttpMultipartParser you can find it's repo at https://github.com/Http-Multipart-Data-Parser/Http-Multipart-Data-Parser

Here's the code you need:

// This method will read HTTP Request
public async void ReadRequestHeader(NetworkStream stream) 
{
    MemoryStream memory = new MemoryStream();
    
    while (stream.DataAvailable && stream.CanRead)
    {
          byte[] b = new byte[256];
          int read = await stream.ReadAsync(b, 0, b.Length);
          await memory.WriteAsync(b, 0, read);
    }
    
    memory.Seek(0, SeekOrigin.Begin);
    byte[] mem = memory.ToArray();
    byte[] emptyline = new byte[] { 13, 10, 13, 10 };

    byte[] data;
    byte[] header;
    bool test = mem.Contains(emptyline, out int endpoint);
    if (test)
    {
         endpoint+= emptyline.Length;
         header = mem[..endpoint]; // your Header bytes
         data = mem[endpoint..];   // your Body bytes

         // if you want to consume http request body. just follow
         MultipartFormDataParser parser = await MultipartFormDataParser.ParseAsync(new MemoryStream(data)).ConfigureAwait(false);
         Console.WriteLine("MyName : {0}", parser.GetParameterValue("username"));
         foreach (FilePart file in parser.Files)
         {
             File.WriteAllBytes(   
    Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), file.FileName), ((MemoryStream)file.Data).ToArray()
);

          }


    }
}

Now, Let's Implement Contains(this byte[] self, byte[] candidate, out int ep)

// Note: I've copied this class from
// https://stackoverflow.com/a/5062250/10903225

    static class ByteArrayRocks
    {

        public static bool Contains(this byte[] self, byte[] candidate, out int ep)
        {
            ep = 0;
            if (IsEmptyLocate(self, candidate))
                return false;

            for (int i = 0; i < self.Length; i++)
            {
                if (IsMatch(self, i, candidate))
                {
                    ep = i;
                    return true;
                }
            }

            return false;
        }

        static bool IsMatch(byte[] array, int position, byte[] candidate)
        {
            if (candidate.Length > (array.Length - position))
                return false;

            for (int i = 0; i < candidate.Length; i++)
                if (array[position + i] != candidate[i])
                    return false;

            return true;
        }

        static bool IsEmptyLocate(byte[] array, byte[] candidate)
        {
            return array == null
                    || candidate == null
                    || array.Length == 0
                    || candidate.Length == 0
                    || candidate.Length > array.Length;
        }
    }

Note: I know it's not the best solution especially if Http request body is too big (Which'll eat up all your memory). but, you can specify fixed number of bytes to be read by MemoryStream

I hope this long solution hepls you.

Answer 2

Here is the important info from your header:

Content-Encoding: gzip

What you're seeing is the gzip data. You need to unzip it.