NetworkStream can read data only after the sending NetworkStream is closed or disposed

Question

I'm writing an app for sending and receiving AES encrypted files. I have two functions, one for sending:

public async Task SendFileAsync()
{
    var buffer = new byte[1024];

    using (Aes aesAlg = Aes.Create())
    {
        // tcpHandler.stream is a NetworkStream
        using (ICryptoTransform encryptor = aesAlg.CreateEncryptor(aesAlg.Key, aesAlg.IV))
        {
            using (Stream fileStream = await selectedFile.OpenStreamForReadAsync())
            {
                using (CryptoStream csEncrypt = new CryptoStream(tcpHandler.stream, encryptor, CryptoStreamMode.Write, true))
                {
                    while (stream.Position < selectedFileSize)
                    {
                        int nowRead = fileStream.Read(buffer, 0, buffer.Length); // read bytes from file
                        csEncrypt.Write(buffer, 0, nowRead); // write bytes to CryptoStream (which writes to NetworkStream)
                    }
                }
            }
        }
    }
    await tcpHandler.stream.FlushAsync()
}

And one for receiving:

public async Task ReceiveFileAsync()
{
    var buffer = new byte[1024];
    BinaryFormatter formatter = new BinaryFormatter();
    int messageLength = tcpHandler.ReadMessageLength();
    int totalBytesRead = 0;

    using (Aes aesAlg = Aes.Create())
    {
        // tcpHandler.stream is a NetworkStream
        using (ICryptoTransform decryptor = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV))
        {
            using (var fileStream = await newFile.OpenStreamForWriteAsync())
            {
                using (CryptoStream csDecrypt = new CryptoStream(tcpHandler.stream, decryptor, CryptoStreamMode.Read, true)) 
                {
                    while (totalBytesRead < messageLength)
                    {
                        // calculate how many bytes have to be read in this iteration
                        var toRead = Math.Min(buffer.Length, messageLength - totalBytesRead);
                        var nowRead = csDecrypt.Read(buffer, 0, toRead); // read bytes from CryptoStream
                        totalBytesRead += nowRead; // sum read bytes
                        fileStream.Write(buffer, 0, nowRead); // write decrypted bytes to file
                    }
                }
            }
        }
    }
}

The issue is that ReceiveFileAsync() blocks itself on the last csDecrypt.Read(buffer, 0, toRead) as if there wasn't enough data in csDecrypt stream. However, when I close (kill the process) the sending application, the receiving application correctly receives the last buffer.

The same thing happens when I change the last parameter of using (CryptoStream csEncrypt = new CryptoStream(tcpHandler.stream, encryptor, CryptoStreamMode.Write, true)) to false - it makes the CryptoStream close the base stream (tcpHandler.stream) when it gets disposed.

If I do tcpHandler.stream.Close() at the end of SendFileAsync() it also helps.

In short, the last buffer I send doesn't get received until I close the sending NetworkStream (tcpHandler.stream), either by closing/disposing it or closing the application.

I tried adding await tcpHandler.stream.FlushAsync() as a last line of SendFileAsync(), but it didn't help. Any ideas what should I do to fix this?

EDIT: Updated code with nested using statements.

Answer 1

My testing .NET 5.0 async console apps with a 5 bytes text file with some sleeping to keep the process alive.

Sender app key points:

1. NetworkStream.FlushAsync or (NetworkStream.Flush)
2. CryptoStream.FlushFinalBlock
3. NetworkStream.Close
4. TcpClient.Close (Also closes the NetworkStream)
5. leaveOpen parameter on the CryptoStream constructor (true -> it leaves the NetworkStream opened, false -> it calls Close on the NetworkStream)

Receiver app key points:

1. leaveOpen parameter on the CryptoStream constructor (nothing interesting here),

SideNotes:

• Flush causes the all data form the buffer to be written to the underlying stream. It does not close the Stream.
• Close obviously closes the stream
• CryptoStream is auto managing the final cipher block itself on stream close so need to do it expectedly with Flushing.

Pretty much you already had the answer for yourself:

You have to close the NetworkStream In order for the receiver to be able to receive it. There are multiple to ways to do it as described.

Flushing does not close the stream, hence it will not work.

This behaviour does not depend on the CryptoStream. I also tried this with a writing directly to the NetworkStream without using encryption -> same results. However you can leave the tcp client open, which means you still will have active connection. So like you discovered and I tested:

• 1), 2) and 5)true will not work (Network stream will be closed)
• 3), 4) and 5)false will work (Network stream will remain opened)

Here is my testing code (almost the same but still decided to post it):

1. The client:

    class Program
    {
        static byte[] key = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16 };
        static byte[] iv = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16 };
   
        static async Task Main(string[] args)
        {
            try
            {
                TcpClient tcpClient = new TcpClient();
                tcpClient.Connect("127.0.0.1", 8001);
                Stream stream = tcpClient.GetStream(); // NetworkStream
   
                await SendFileAsync(stream, "test.txt");
   
                //await stream.FlushAsync();
   
                //stream.Close();
   
                //tcpClient.Close();
                Thread.Sleep(10000);
            }
   
            catch (Exception e)
            {
                Console.WriteLine("Error..... " + e.Message);
            }
        }
   
        public static async Task SendFileAsync(Stream stream, string filePath)
        {
            var buffer = new byte[1024];
   
            using (Aes aesAlg = Aes.Create())
            using (ICryptoTransform encryptor = aesAlg.CreateEncryptor(key, iv))//aesAlg.Key, aesAlg.IV
            {
                using (FileStream fs = new FileStream(filePath, FileMode.Open))
                using (BinaryReader br = new BinaryReader(fs))
                {
                    using (CryptoStream csEncrypt = new CryptoStream(stream, encryptor, CryptoStreamMode.Write, false))
                    {
                        int bytesRead;
                        while ((bytesRead = br.Read(buffer, 0, buffer.Length)) > 0)
                        {
                            csEncrypt.Write(buffer, 0, bytesRead);
                            //csEncrypt.FlushFinalBlock();
                            break;
                        }
                    }
                }
            }
        }
    }
   

2. The server:

    class Program
    {
        static byte[] key = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16 };
        static byte[] iv = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16 };
   
        public static async Task Main(string[] args)
        {
            try
            {
                IPAddress ipAd = IPAddress.Parse("127.0.0.1");
                TcpListener tcpServer = new TcpListener(ipAd, 8001);
                tcpServer.Start();
                var tcpClient = await tcpServer.AcceptTcpClientAsync();
   
                await ReceiveFileAsync(tcpClient, "text.txt");
   
                tcpServer.Stop();
            }
            catch (Exception e)
            {
                Console.WriteLine("Error..... " + e.Message);
            }
        }
   
        public static async Task ReceiveFileAsync(TcpClient tcpClient, string filePath)
        {
            var buffer = new byte[1024];
            var stream = tcpClient.GetStream();
   
            using (Aes aesAlg = Aes.Create())
            using (ICryptoTransform decryptor = aesAlg.CreateDecryptor(key, iv))
            {
                using (FileStream fs = new FileStream(filePath, FileMode.Create))
                using (BinaryWriter br = new BinaryWriter(fs))
                {
                    using (CryptoStream csDecrypt = new CryptoStream(stream, decryptor, CryptoStreamMode.Read, true))
                    {
                        int bytesRead;
                        while ((bytesRead = csDecrypt.Read(buffer, 0, buffer.Length)) > 0)
                        {
                            br.Write(buffer, 0, bytesRead);
                        }
                    }
                }
            }
        }
    }