Message struct for tcp/ip data transfer in C#

Question

long time ago I found usefull guide about creating and managing message struct classes for client/server purposes but I can't find it again.

It must contain 4 fields:

_FLAG (integer from 0 to 1000)
_FROM (string up to 16 chars)
_TO (same as above)
_DATA (string up to 48 chars)

And I want to be able to easily convert it to byte array, send through socket, then receive, convert it to struct and read. What's the best way to do that?

EDIT

Thanks for your answers (maybe I wasn't clear enough) but it wasn't what I was looking for. I've just found something usefull and made it this way:

    [MarshalAs(UnmanagedType.U4)]
    public uint _flag;

    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
    public String _from;

    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
    public String _to;

    [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 20)]
    public String _data;

    public byte[] Serialize()
    {
        var buffer = new byte[Marshal.SizeOf(typeof(DataPacket))];
        var gch = GCHandle.Alloc(buffer, GCHandleType.Pinned);
        var pBuffer = gch.AddrOfPinnedObject();
        Marshal.StructureToPtr(this, pBuffer, false);
        gch.Free();
        return buffer;
    }

    public void Deserialize(ref byte[] data)
    {
        var gch = GCHandle.Alloc(data, GCHandleType.Pinned);
        this = (DataPacket)Marshal.PtrToStructure(gch.AddrOfPinnedObject(), typeof(DataPacket));
        gch.Free();
    }

Answer 1

If you use one of existing serializer classes, they come with some expenses. You can create your own serializer that address only your needs. There are always trade-offs between using something standard and your own implementation. Here's the example I created for one of my projects. User CreateFromBytes and StoreToBytes to get or restore from the byte array. The stream versions are good if you work with stream (that also might work in your case) and it helps to avoid additional transformation.

// well, there's a bad naming for this class as T can be of any type.
public abstract class ClassSerializer<T> : IClassSerializer<T> 
{
    public const int VersionNull = 0;

    public abstract int StoreToStream(Stream stream, T item);
    public abstract T CreateFromStream(Stream stream);

    public virtual byte[] StoreToBytes(T item)
    {
        byte[] result;
        using (var stream = new MemoryStream(16))
        {
            StoreToStream(stream, item);
            result = stream.ToArray();
        }
        return result;
    }

    public virtual T CreateFromBytes(byte[] bytes)
    {
        using (var stream = new MemoryStream(bytes))
        {
            return CreateFromStream(stream);
        }
    }

    public int WriteVersionNull(Stream stream)
    {
        return stream.WriteIntVariableLength(VersionNull);
    }
}

For particular class (structure in your case; I recommend you to go with classes; you don't get much benefits working with structures; anyway it is converted to a byte array) you should create a particular serializer like this.

public class DescriptionSerializer : ClassSerializer<Description>
{

    private const byte Version = 1;

    public override int StoreToStream(Stream stream, KeyDescription item)
    {
        if (item == null)
        {
            return WriteVersionNull(stream);
        }
        var count = stream.WriteIntVariableLength(Version);
        count += stream.WriteString(item.StringProperty);
        count += stream.WriteBytes(item.BytesPropery);
        count += stream.WriteIntVariableLength(item.IntegerProperty);

        return count;
    }

    public override Description CreateFromStream(Stream stream)
    {
        var version = stream.ReadIntVariableLength();
        if (version == VersionNull) return null;
        if (version != Version)
        {
            throw new InvalidDataException(
                string.Format("The stream version '{0}' is incorrect. The data cannot be deserialized. ", version));
        }
        var stringProperty = stream.ReadString();
        var bytesProperty = stream.ReadBytes();
        var integerProperty = stream.ReadIntVariableLength();

        return new Description(
            stringProperty, 
            bytesProperty, 
            integerProperty);
    }
}

And you need a helper class that can write different basic types to a stream.

public static class SerializationHelper
{
    public static int WriteArray<T>(
        [NotNull] this Stream stream, 
        [NotNull] IClassSerializer<T> serializer, 
        [CanBeNull] T[] items ) where T: class
    {
        if (items == null)
        {
            return stream.WriteIntVariableLength(0);
        }
        var count = stream.WriteIntVariableLength(items.Length);
        foreach (var item in items)
        {
            count += serializer.StoreToStream(stream, item);
        }
        return count;
    }

    public static T[] ReadArray<T>(
        [NotNull] this Stream stream,
        [NotNull] IClassSerializer<T> serializer) where T : class
    {
        var length = stream.ReadIntVariableLength();
        if (length <= 0)
        {
            return new T[0];
        }
        var result = new T[length];
        for (int i = 0; i < length; i++)
        {
            var item = serializer.CreateFromStream(stream);
            result[i] = item;
        }
        return result;
    }

    public static int WriteString([NotNull] this Stream stream, [CanBeNull] string item)
    {
        if (item == null)
        {
            return stream.WriteIntVariableLength(0);
        }
        var buf = Encoding.UTF8.GetBytes(item);
        var count = stream.WriteIntVariableLength(buf.Length);
        stream.Write(buf, 0, buf.Length);
        return count + buf.Length;
    }

    public static string ReadString([NotNull] this Stream stream)
    {
        var len = stream.ReadIntVariableLength();
        if (len <= 0) return null;
        var buf = new byte[len];
        var count = stream.Read(buf, 0, len);
        if (count != buf.Length)
        {
            throw new InvalidDataException("There should be more bytes to read.");
        }
        var result = Encoding.UTF8.GetString(buf);
        return result;
    }

    public static int WriteIntVariableLength([NotNull] this Stream stream, Int32 value)
    {
        var len = unchecked((UInt32)value);
        if (len < 0x80)
        {
            stream.Write(new [] { (byte) (len & 0x7F) }, 0, 1);
            return 1;
        }

        if (len < 0x4000)
        {
            stream.Write(new[] { (byte)((len & 0x7F) | 0x80) }, 0, 1);
            stream.Write(new[] { (byte)(((len>>7) & 0x7F)) }, 0, 1);
            return 2;
        }
        if (len < 0x200000)
        {
            stream.Write(new[] { (byte)((len & 0x7F) | 0x80) }, 0, 1);
            stream.Write(new[] { (byte)(((len >> 7) & 0x7F) | 0x80) }, 0, 1);
            stream.Write(new[] { (byte)((len >> 14) & 0x7F) }, 0, 1);
            return 3;
        }

        if (len < 0x10000000)
        {
            stream.Write(new[] { (byte)((len & 0x7F) | 0x80) }, 0, 1);
            stream.Write(new[] { (byte)(((len >> 7) & 0x7F) | 0x80) }, 0, 1);
            stream.Write(new[] { (byte)(((len >> 14) & 0x7F) | 0x80)  }, 0, 1);
            stream.Write(new[] { (byte)(((len >> 21) & 0x7F)) }, 0, 1);
            return 4;
        }

        stream.Write(new[] { (byte)((len & 0x7F) | 0x80) }, 0, 1);
        stream.Write(new[] { (byte)(((len >> 7) & 0x7F) | 0x80) }, 0, 1);
        stream.Write(new[] { (byte)(((len >> 14) & 0x7F) | 0x80) }, 0, 1);
        stream.Write(new[] { (byte)(((len >> 21) & 0x7F) | 0x80) }, 0, 1);
        stream.Write(new[] { (byte)(((len >> 28) & 0x0F)) }, 0, 1);
        return 5;
    }

    public static Int32 ReadIntVariableLength([NotNull] this Stream stream)
    {
        //var buf = new byte[sizeof(Int32)];
        var b1 = stream.ReadByte();
        if (b1==-1) throw new InvalidDataException("Unexpected end of the stream");
        if ((b1 & 0x80) == 0)
        {
            return b1;
        }
        var b2 = stream.ReadByte();
        if (b2==-1) throw new InvalidDataException("Unexpected end of the stream");
        if ((b2 & 0x80) == 0)
        {
            return (b1 & 0x7F) +  ((b2 & 0x7F) <<7);
        }
        var b3 = stream.ReadByte();
        if (b3==-1) throw new InvalidDataException("Unexpected end of the stream");
        if ((b3 & 0x80) == 0)
        {
            return (b1 & 0x7F) +  ((b2 & 0x7F) <<7) + ((b3 & 0x7F) <<14);
        }
        var b4 = stream.ReadByte();
        if (b4==-1) throw new InvalidDataException("Unexpected end of the stream");
        if ((b4 & 0x80) == 0)
        {
            return (b1 & 0x7F) +  ((b2 & 0x7F) <<7) + ((b3 & 0x7F) <<14) + ((b4 & 0x7F) <<21);
        }
        var b5 = stream.ReadByte();
        if (b5 == -1) throw new InvalidDataException("Unexpected end of the stream");
        return (b1 & 0x7F) + ((b2 & 0x7F) << 7) + ((b3 & 0x7F) << 14) + ((b4 & 0x7F) << 21) + ((b5 & 0x0F) << 28);
    }

    public static int WriteBytes([NotNull] this Stream stream, [CanBeNull]byte[] bytes)
    {
        if (bytes == null)
        {
            return stream.WriteIntVariableLength(0);
        }
        var length = bytes.Length;
        var count = stream.WriteIntVariableLength(length);
        stream.Write(bytes, 0, length);
        return count + length;
    }

    public static byte[] ReadBytes([NotNull] this Stream stream)
    {
        var length = stream.ReadIntVariableLength();
        if (length == 0) return new byte[0];
        var bytes = new byte[length];
        var count = stream.Read(bytes, 0, length);
        if (count != length)
        {
            throw new InvalidDataException("There should be more bytes");
        }
        return bytes;            
    }
}

You can use serializer classes in StoreToStream and CreateFromStream methods to serialize other types, like this:

    public override int StoreToStream(Stream stream, SomeClass item)
    {
        if (item == null)
        {
            return WriteVersionNull(stream);
        }
        var count = stream.WriteIntVariableLength(Version);
        count += new SubType1Serializer().StoreToStream(stream, item.SubType1);
        count += new SubType2Serializer().StoreToStream(stream, item.SubType2);

I hope the idea is clear though I don't provide a detailed description.