Deserialize byte[] with blank at the end

Question

I am trying to send a serialized object through a network tunnel, and while sending it, it adds whitespace (blank) at the end of the byte[], so it fits the selected size. When I go to deserialize it, it throws an error:

SerializationException: End of Stream encountered before parsing was completed

Code:

[Serializable]
class Foo
{
    public int number;
    public string str;
}

public class ExampleServer
{
    public static void Main(string[] args)
    {
        TcpListener listener = new TcpListener(9000);
        listener.Start();
        TcpClient serverClient = listener.AcceptTcpClient();

        IFormatter formatter = new BinaryFormatter();
        Stream stream = new MemoryStream();

        byte[] buffer = new byte[2048];
        serverClient.GetStream().Read(buffer, 0, buffer.Length);

        stream.Write(buffer, 0, buffer.Length);

        Foo fo = (Foo)formatter.Deserialize(stream);

        Console.WriteLine(fo.number);
    }
}

public class ExampleClient
{
    public static void Main(string[] args)
    {
        TcpClient client = new TcpClient();
        client.Connect("127.0.0.1", 9000);

        Foo fo = new Foo();
        fo.number = 10;
        fo.str = "str";

        IFormatter formatter = new BinaryFormatter();
        Stream stream = new MemoryStream();

        formatter.Serialize(stream, fo);

        byte[] buffer = new byte[2048];

        stream.Read(buffer, 0, buffer.Length);

        client.GetStream().Write(buffer, 0, buffer.Length);

    }
}

How would I fix this?

Done. It's there. The Foo class is in a library, and the different class in separate scripts. — decduck3, Aug 22 '20 at 22:51
Look at the documentation of `Read`, it returns a `int` any code that calls read without looking at that int has a bug in it. Remember one write on the server does not have to equal one read on the client. It could end up being multiple reads or even having two writes combined together. Learn about [Message Framing](https://blog.stephencleary.com/2009/04/message-framing.html), that is how you solve this problem with network streams. — Scott Chamberlain, Aug 22 '20 at 23:03
You have a bug in both implementations by assuming that Read fills the buffer. You need to use the return value (bytes read) and only write that many bytes (which could be buffer.Length or 17 or 2046 or even zero) — pinkfloydx33, Aug 22 '20 at 23:32
Can you change the wire format, or is it fixed? If you can, you really need to implement message framing, see e.g. https://blog.stephencleary.com/2009/04/message-framing.html or [Sending and receiving data over a network using TcpClient](https://stackoverflow.com/q/3609280) or [Problem with Socket and Serialization C#](https://stackoverflow.com/q/3111384). For instance, you may not receive the complete contents of the binary stream upon your first call to `Read()`. — dbc, Aug 22 '20 at 23:52
Oof, where to start. That stream read code is just broken (you aren't checking the result); BinaryFormatter is actively harmful and discouraged by just about everyone who deals with serialization, you don't seem to be rewinding between write (serialize) and read. The immediate bug is probably that last one: rewinding will probably fix it, but: oof, this is not advisable code — Marc Gravell, Aug 22 '20 at 23:56

score 1 · Accepted Answer · answered Aug 23 '20 at 07:10

In both code samples, the fundamental problem is that you write a buffer to a MemoryStream and then immediately try to read from it, without rewinding. MemoryStream is like a video tape: it only has one position, and if you don't rewind: you're in the wrong place. So fixing that will probably help.

However, that is not the only problem:

when reading from the network stream, you don't check the return of Read. That means that all you have is at least one byte. Calling Read with a buffer does not fill the buffer - it blocks until it can read at least one byte, or the end of the stream is detected; you must check the return value of Read
technically this also applies to MemoryStream, although in reality you will get away with it there
sending / expecting 2k is ... bizarre; send what the payload size is, not something arbitrarily oversized; you probably need to know about "framing", which in a binary format usually means: length-prefix
perhaps ironically, BinaryFormatter does framing itself anyway; if you'd just pointed BinaryFormatter at the network stream, it probably would have worked, but:
BinaryFormatter is a terrible choice, frankly; I could write many paragraphs explaining why - but I already did

As general advice: network code is hard. If you aren't an expert in that area, I'd strongly recommend using a pre-rolled RPC or similar layer.

Thanks for this. I am not really an expert on networking, but I know quite a bit. I just picked 2KB because I didn't know about message framing, and it was easier to do it that way. Also, instead of BinaryFormatter, would I use my own serialization? Or are there alternatives that I don't know of? — decduck3, Aug 24 '20 at 04:44
@decduck3 that depends what your primary aims are; I'm partial to protobuf-net, but I'm also biased since I'm the author and maintainer, but ultimately, for things that aren't performance / bandwidth critical, just about anything should work - Json.Net (Newtonsoft), XmlSerializer, DataContractSerializer, etc — Marc Gravell, Aug 24 '20 at 08:23

score 0 · Answer 2 · answered Aug 22 '20 at 22:29

Try to add [Serializable] tag to the class you are Serializing and check if this solve you problem.

add like this:

[Serializable]
public class TestSimpleObject  {

    public int member1;
    public string member2;
    public string member3;
    public double member4;
}

Deserialize byte[] with blank at the end

2 Answers2