How to write to MemoryStream if many messages are sent

Question

Here is how i read data from my stream now:

public List<ServerClient> clients = new List<ServerClient>();

while (true)
{
    Update();
}

    private void Update()
    {
        //Console.WriteLine("Call");
        if (!serverStarted)
        {
            return;
        }

        foreach (ServerClient c in clients.ToList())
        {
            // Is the client still connected?
            if (!IsConnected(c.tcp))
            {
                c.tcp.Close();
                disconnectList.Add(c);
                Console.WriteLine(c.connectionId + " has disconnected.");
                CharacterLogout(c.connectionId);
                continue;
                //Console.WriteLine("Check for connection?\n");
            }
            else
            {



                // Check for message from Client.
                NetworkStream s = c.tcp.GetStream();
                if (s.DataAvailable)
                {
                    string data = c.streamReader.ReadLine();

                    if (data != null)
                    {
                        OnIncomingData(c, data);
                    }

                }
                //continue;
            }
        }

        for (int i = 0; i < disconnectList.Count - 1; i++)
        {
            clients.Remove(disconnectList[i]);
            disconnectList.RemoveAt(i);
        }


    }

When data is read it is send to OnIncomingData function which is processing the data. I don't have problems there.

Here is how i send data to the stream:

public void Send(string header, Dictionary data) {

if (stream.CanRead)
{
    socketReady = true;
}

if (!socketReady)
{
    return;
}
JsonData SendData = new JsonData();
SendData.header = "1x" + header;
foreach (var item in data)
{
    SendData.data.Add(item.Key.ToString(), item.Value.ToString());
}
SendData.connectionId = connectionId;

string json = JsonConvert.SerializeObject(SendData);
var howManyBytes = json.Length * sizeof(Char);
writer.WriteLine(json);
writer.Flush();

Debug.Log("Client World:" + json);

}

Here is my:

public class ServerClient
{
    public TcpClient tcp;
    public int accountId;
    public StreamReader streamReader;
    public int connectionId;
    public ServerClient(TcpClient clientSocket)
    {
        tcp = clientSocket;
    }
}

Here is my OnConnection function:

    private void OnConnection(IAsyncResult ar)
    {
        connectionIncrementor++;
        TcpListener listener = (TcpListener)ar.AsyncState;
        NetworkStream s = clients[clients.Count - 1].tcp.GetStream();
        clients.Add(new ServerClient(listener.EndAcceptTcpClient(ar)));
        clients[clients.Count - 1].connectionId = connectionIncrementor;
        clients[clients.Count - 1].streamReader = new StreamReader(s, true);
        StartListening();


        //Send a message to everyone, say someone has connected!
        Dictionary<string, string> SendDataBroadcast = new Dictionary<string, string>();
        SendDataBroadcast.Add("connectionId", clients[clients.Count - 1].connectionId.ToString());

        Broadcast("001", SendDataBroadcast, clients, clients[clients.Count - 1].connectionId);
        Console.WriteLine(clients[clients.Count - 1].connectionId + " has connected.");
    }

Normally everything works fine. However if i try to send more request per 1 second the problem occurs. The message received is not full and complete. It just receives a portion of the message sent.

From Debug.Log("Client World:" + json); i can see that the message is full and complete but on the server i see that it is not.

This is not happening if i send less requests.

So for that reason i think i should create a MemoryStream and puts a message there and read it after. However i'm really not sure how i can do that. Can you help ?

Answer 1

The whole code is not very good, but I'll concentrate on your specific problem. It's most likely related to data buffering by StreamReader. StreamReader has buffer size (which you can pass to constructor) which defaults to 1024 bytes. When you call ReadLine - it's perfectly possible for stream reader to read more than one line from the underlying stream. In your case - you have while loop in which you enumerate connected clients and in every iteration of the loop you create new StreamReader and read one line from it. When message rate is low - all looks fine, because between your loop iterations only one line arrives. Now suppose client quickly sent 2 json messages, each of which is 800 bytes, and they both arrived into your socket. Now you call StreamReader.ReadLine. Because buffer size is 1024 - it will read 1024 bytes from socket (NetworkStream) and return first 800 to you (as a line). You process that line and discard StreamReader going to the next iteration of your while loop. By doing that you also discard part of the message (224 bytes of the next message), because they were already read from the socket into StreamReader buffer. I think from that it should be clear how to solve that problem - don't create new StreamReader every time but create one per client (for example store as a member of ServerClient) and use that.

Answer 2

The client looks more suspicious to me than the server.

StreamWriter is not thread-safe. Are you calling it in a thread-safe manner when using ClientWorldServer.Send? Lock up or queue your calls to ClientWorldServer.Send using a lock or BlockingCollection or some other synchronisation primitive. There is also a thread-safe wrapper of streamwriter you might be able to use.