NamedPipeServerStream receive MAX=1024 bytes, why?

Question

I'm using a NamedPipeStream, client and server, I'm sending data from client to server, the data is a serialize object which include binary data.

When the server side receive the data, it always have MAX 1024 size while client send much more!! so when try to serialize the data, this cause the following exception: "Unterminated string. Expected delimiter: ". Path 'Data', line 1, position 1024."

the server buffer size defined as:

protected const int BUFFER_SIZE = 4096*4;
var stream = new NamedPipeServerStream(PipeName,
                                                   PipeDirection.InOut,
                                                   1,
                                                   PipeTransmissionMode.Message,
                                                   PipeOptions.Asynchronous,
                                                   BUFFER_SIZE,
                                                   BUFFER_SIZE,
                                                   pipeSecurity);


        stream.ReadMode = PipeTransmissionMode.Message;

I'm using :

    /// <summary>
    /// StreamWriter for writing messages to the pipe.
    /// </summary>
    protected StreamWriter PipeWriter { get; set; }

The read function:

/// <summary>
/// Reads a message from the pipe.
/// </summary>
/// <param name="stream"></param>
/// <returns></returns>
protected static byte[] ReadMessage(PipeStream stream)
{
    MemoryStream memoryStream = new MemoryStream();

    byte[] buffer = new byte[BUFFER_SIZE];

    try
    {
        do
        {
            if (stream != null)
            {
                memoryStream.Write(buffer, 0, stream.Read(buffer, 0, buffer.Length));
            }

        } while ((m_stopRequested != false) && (stream != null) && (stream.IsMessageComplete == false));
    }
    catch
    {
        return null;
    }
    return memoryStream.ToArray();
}


protected override void ReadFromPipe(object state)
{
    //int i = 0;
    try
    {
        while (Pipe != null && m_stopRequested == false)
        {
            PipeConnectedSignal.Reset();

            if (Pipe.IsConnected == false)
            {//Pipe.WaitForConnection();
                var asyncResult = Pipe.BeginWaitForConnection(PipeConnected, this);

                if (asyncResult.AsyncWaitHandle.WaitOne(5000))
                {
                    if (Pipe != null)
                    {
                        Pipe.EndWaitForConnection(asyncResult);
                        // ...
                        //success;
                    }
                }
                else
                {
                    continue;
                }
            }
            if (Pipe != null && Pipe.CanRead)
            {
                byte[] msg = ReadMessage(Pipe);

                if (msg != null)
                {
                    ThrowOnReceivedMessage(msg);
                }
            }
        }
    }
    catch (System.Exception ex)
    {
        System.Diagnostics.Debug.WriteLine(" PipeName.ReadFromPipe Ex:" + ex.Message);
    }
}

I don't see in the client side where I can define or change the buffer size!

Any idea?!

Answer 1

The basic problem is that you didn't read enough. You need to repeat the read operation if PipeStream.IsMessageComplete is false, and keep doing that until it returns true - that tells you the whole message has been read. Depending on your deserializer, you might need to store the data in a buffer of your own, or create some wrapper stream to handle this for you.

A simple example of how this could work for simple string deserialization:

void Main()
{
  var serverTask = Task.Run(() => Server()); // Just to keep this simple and stupid

  using (var client = new NamedPipeClientStream(".", "Pipe", PipeDirection.InOut))
  {
    client.Connect();
    client.ReadMode = PipeTransmissionMode.Message;

    var buffer = new byte[1024];
    var sb = new StringBuilder();

    int read;
    // Reading the stream as usual, but only the first message
    while ((read = client.Read(buffer, 0, buffer.Length)) > 0 && !client.IsMessageComplete)
    {
      sb.Append(Encoding.ASCII.GetString(buffer, 0, read));
    }

    Console.WriteLine(sb.ToString());
  }
}

void Server()
{
  using (var server
    = new NamedPipeServerStream("Pipe", PipeDirection.InOut, 1, 
                                PipeTransmissionMode.Message, PipeOptions.Asynchronous)) 
  {
    server.ReadMode = PipeTransmissionMode.Message;      
    server.WaitForConnection();

    // On the server side, we need to send it all as one byte[]
    var buffer = Encoding.ASCII.GetBytes(File.ReadAllText(@"D:\Data.txt"));
    server.Write(buffer, 0, buffer.Length); 
  }
}

Just as a side-note - I can easily read or write as much data as I want at once - the limiting factor is the buffer I use, not the one the pipes use; although I'm using a local named pipe, it might be different for a TCP pipe (though it'd be a bit annoying - it's supposed to be abstracted away from you).

EDIT:

Okay, now it's finally obvious what your problem is. You can't use a StreamWriter - when sending a message long enough, it will result in multiple Write calls on the pipe stream, resulting in multiple separate message for your data. If you want the whole message as a single message, you must use a single Write call. For example:

var data = Encoding.ASCII.GetBytes(yourJsonString);
Write(data, 0, data.Length);

The 1024-long buffer is StreamWriters, it has nothing to do with the named pipes. Using StreamWriter/StreamReader is a bad idea in any networking scenario, even when working with raw TCP streams. It just isn't what it's designed for.