C# Deserializing a struct after receiving it through TCP

Question

I am sending my own struct "packet" object through the TCP interface that C# offers with TCPListener and TCPClient.

This is my struct

[Serializable]
struct RemuseNetworkPacket
{
    public String ApplicationCode;
    public String ReceiverCode;
    public RemusePacketType Type;
    public uint ID;
    public uint cID;
    public String Name;
    public byte[] Data;
    public String Text;
    public System.Drawing.Point Coords;
    public String Timestamp;
    public String Time;
    public String SenderName;

    public byte[] Serialize()
    {
        var buffer = new byte[Marshal.SizeOf(typeof(RemuseNetworkPacket))];
        var gch = GCHandle.Alloc(buffer, GCHandleType.Pinned);
        var pBuffer = gch.AddrOfPinnedObject();
        Marshal.StructureToPtr(this, pBuffer, false);
        gch.Free();
        return buffer;
    }

    public void Deserialize(byte[] data)
    {
        var gch = GCHandle.Alloc(data, GCHandleType.Pinned);
        this = (RemuseNetworkPacket)Marshal.PtrToStructure(gch.AddrOfPinnedObject(), typeof(RemuseNetworkPacket));
        gch.Free();
    }
}

I am using the serialization methods within the struct to prepare and retrieve the data before and after sending.

In order to to let the receiver know the incoming data's size, I add some header data to the bytes being sent, in the format of l=212;... which means length = 212; and the ... is the rest of the packet.

On the receiving end I search for this until I find the entire l=xxxx; then I make a new byte array without the header, then attempt to deserialize the data. The packet byte to use for deserialization is: tcp stream's buffer.Length - foundHeaderSize (l=xxxx;)

If I run the client and server on the same machine, it works without errors, however if I have the client and server on separate machines, I get exceptions and it crashes.

The exception takes place when the packet is being deserialized, saying:

*System.Runtime.InteropServices.SafeArrayTypeMismatchException Mismatch has occurred between the runtime type of the array and the sb type recorded in the metadata at System.Runtime.InteropServices.PtrToStructureHelper

Stacktrace: System.Runtime.InteropServices.PtrToStructureHelper (IntPtr ptr, Object structure, Boolean allowValueClasses) at System.Runtime.InteropServices.PtrToStructure(IntPtr ptr, Type structureType..*

I'm asking for help to identify the cause of the problem. Can I not do it like this with objects that came over the network?

have you checked that you run 32/64 bit on both systems? Do the .NET versions match? I would not use this cross network. Protobuf is also pretty fast and it avoids such issues. — weismat, May 20 '16 at 08:16
Why are you subtracting the buffer size with the packet size?? That does not seem to return the packet correctly. Add a four-byte header to the beginning of every packet instead, no other chars. Then you just take those four bytes out when reading/deserializing it. — Visual Vincent, May 20 '16 at 08:23
It's the same OS and 64 bit on both systems. I am also using the same application on both machines. — Alx, May 20 '16 at 08:25
Alternatively use real serialization, such as binary serialization. As weismat says, the IntPtr structure differ on 32- and 64-bit machines. On 32-bit it's a normal `Int32`, but on 64-bit machines it's an `Int64` (aka `Long`). --- You can also see [**this answer of mine**](http://stackoverflow.com/a/35240061/3740093), which has a solution for separating packets by using their length as a header. — Visual Vincent, May 20 '16 at 08:28
@VisualVincent I thought I had to do it like this in order to separate the added string " l=xxx; " with the bytes that represent the packet object. — Alx, May 20 '16 at 08:28
@VisualVincent Thanks for the link, I will take a look at your previous answer and see if I can improve my structure. — Alx, May 20 '16 at 08:30
In order to separate the string you would have to subtract the length of the string (the amount of bytes), not the entire buffer. But the classes in the answer I gave you will separate packets and take the header out for you, so the only thing you would need to do is to handle the data that comes out of it. :) — Visual Vincent, May 20 '16 at 08:32
@VisualVincent The method I used to get the headersize up until now is that I get the .Length from the "l=xxx;" string (xxx being whatever number was found. Then subtract this headersize from the buffer to be left with the bytes of the Packet struct only. — Alx, May 20 '16 at 08:46
@VisualVincent I was able to make a much improved and working packet, also sending, receiving and deserializing properly thanks to your examples. I can't accept a comment as the answer (I think), so if you post an answer with the link I'll set is as the accepted answer for my problem :) — Alx, May 20 '16 at 13:19

Visual Vincent · Accepted Answer · 2017-10-15T08:10:17.330

Instead of having a string represent your packet length and then subtract by the string's length to know where to start reading, you should implement proper length-prefixing. Length-prefixing combined with a data header will make you able to read every packet according to its size, then the data header will help you determine what to do with the data.

Ordinary length-prefixing adds a fixed header to every "packet" you send. To create this header you convert an integer (the length of your data) to bytes, which will result in 4 bytes, then you add the data header after that and also the rest of the packet (which is the data you want to send).

This will create the following packet structure:

[Length (4 bytes)][Header (1 byte)][Data (x byte(s))]

Reading a packet is very simple:

Read the first 4 bytes (Length), convert and assign them to an integer variable.
Read the next byte (the data header) and put that in a variable.
Read x bytes to a byte array (where x is the integer you declared in step 1).
Use the data header from step 2 to determine what to do with your data (the byte array from step 3).

In one of my previous answers you can see an example of what I just explained above.

@MadsM : No problem! Glad I could help! – Visual Vincent May 22 '16 at 15:55 — Visual Vincent, May 22 '16 at 15:55

score 0 · Answer 2 · answered May 20 '16 at 08:22

0

Serialized binary data of a structure might differ depending on the platform and OS. e.g. different alignment, different size of long That might be the reason why your code works on the same machine, but not on a different machine.

I would suggest you either use a library like Google ProtoBuf (fast) https://developers.google.com/protocol-buffers/docs/csharptutorial

or rely on the C# object serialization using e.g. XML Serialization (slow) https://msdn.microsoft.com/en-us/library/58a18dwa(v=vs.110).aspx

answered May 20 '16 at 08:22

illfang

65
1
4

Changing a structure to an IntPtr is not really serialization though, more like conversion. Real serialization is for example XML Serialization or Binary Serialization. -- And for sending through the network, I believe Binary Serialization would be the best choice. – Visual Vincent May 20 '16 at 08:36

C# Deserializing a struct after receiving it through TCP

2 Answers2

Linked