A way to decrease memory usage when storing byte array

Question

I'm a beginner learning sockets for a few days now

private void SendFile(Socket client, string user, string folder1, string folder2, string zipFile)
{
    byte[] zipBytes = File.ReadAllBytes(zipFile); //file

    string fileInfoStr = user + ","
   + folder1 + ","
   + folder2 + ","
   + Path.GetFileName(zipFile) + ","
   + zipBytes.Length;

    byte[] fileInfo = Encoding.UTF8.GetBytes(fileInfoStr);

    byte[] fileInfoLen = BitConverter.GetBytes(fileInfo.Length);
    var clientData = new byte[4 + fileInfo.Length + zipBytes.Length];

    fileInfoLen.CopyTo(clientData, 0);
    fileInfo.CopyTo(clientData, 4);
    zipBytes.CopyTo(clientData, 4 + fileInfo.Length);

    // Begin sending the data to the remote device.  
    client.BeginSend(clientData, 0, clientData.Length, 0,
new AsyncCallback(SendCallback), client);

    Receive(client);
}

The problem is for example I try to send a large zip file(300mb), the memory usage shoots up to 600mb+. I'm stuck on how can I decrease the memory usage. Thanks.

You're reading the entire 300MB file into memory with `File.ReadAllBytes(zipFile)`, then you're allocating *another* 300MB byte array (plus a few more bytes) with `new byte[4 + fileInfo.Length + zipBytes.Length]`. You would have to read the file in smaller blocks and send those one at a time to prevent so much memory usage. — Herohtar, Nov 16 '19 at 10:08
Does this answer your question? [Read large files](https://stackoverflow.com/questions/49256316/read-large-files) or [How do I use File.ReadAllBytes In chunks](https://stackoverflow.com/q/11597295/150605) — Lance U. Matthews, Nov 16 '19 at 18:30

score 1 · Accepted Answer · edited Nov 16 '19 at 17:13

You decrease memory usage by not using memory.

Like not using statements like this:

byte[] zipBytes = File.ReadAllBytes(zipFile);

Instead you use FileStream like methods :

const int BufSize = 32 * 1024; // 32K you can also increase this
using (var fileStream = new FileStream(zipFile, FileMode.Open))
{
    using (var reader = new BinaryReader(fileStream))
    {
        var chunk = new byte[BufSize];
        var ix = 0;
        var read = 0;
        while ( (read = reader.Read(chunk,ix,BufSize))>0)
        {
            ix += read;
            client.Send(chunk);
        }
    }
}

You read chunks into memory, not the entire file. And send it with in chunks.

A way to decrease memory usage when storing byte array

1 Answers1