Memory leak with an array c#

Question

User specifies filename and block size. Original file splits into blocks with users block size (except last block). For each block calculates hash-function SHA256 and writes to the console.

This is program with 2 threads: first thread reading the original file and put into queue byte array of block; second thread removes byte array of block from queue and calculate hash.

After first iteration memory doesn't dispose until the program complete.
On next iterations memory allocates and disposes normally.
So, during next reading of part array I get OutOfMemoryException.

How can I manage memory correctly to avoid memory leak?

class Encryption
{
    static FileInfo originalFile;
    static long partSize = 0; 
    static long lastPartSize = 0; 
    static long numParts = 0; 
    static int lastPartNumber = 0; 
    static string[] hash; 
    static Queue<byte[]> partQueue = new Queue<byte[]>();

    public Encryption(string _filename, long _partSize)
    {
        try
        {
            originalFile = new FileInfo(@_filename);
            partSize = _partSize;

            numParts = originalFile.Length / partSize; 
            lastPartSize = originalFile.Length % partSize; 

            if (lastPartSize != 0)
            {
                numParts++;
            }
            else if (lastPartSize == 0)
            {
                lastPartSize = partSize;
            }

            lastPartNumber = (int)numParts - 1;

            hash = new string[numParts];
        }
        catch (FileNotFoundException fe)
        {
            Console.WriteLine("Error: {0}\nStackTrace: {1}", fe.Message, fe.StackTrace);
            return;
        }
        catch (Exception e)
        {
            Console.WriteLine("Error: {0}\nStackTrace: {1}", fe.Message, fe.StackTrace);
        }
    }

    private void readFromFile()
    {
        try
        {
            using (FileStream fs = new FileStream(originalFile.FullName, FileMode.Open, FileAccess.Read))
            {
                for (int i = 0; i < numParts; i++)
                {                        
                    long len = 0;

                    if (i == lastPartNumber)
                    {
                        len = lastPartSize;
                    }
                    else
                    {
                        len = partSize;
                    }                            

                    byte[] part = new byte[len];                

                    fs.Read(part, 0, (int)len);

                    partQueue.Enqueue(part);

                    part = null;
                }
            }
        }   
        catch(Exception e)
        {
            Console.WriteLine("Error: {0}\nStackTrace: {1}", fe.Message, fe.StackTrace);
        }    
    }

    private static void hashToArray()
    {
        try
        {
            SHA256Managed sha256HashString = new SHA256Managed(); 
            int numPart = 0;

            while (numPart < numParts)
            {
                long len = 0;
                if (numPart == lastPartNumber)
                {
                    len = lastPartSize;
                }
                else
                {
                    len = partSize;
                }

                hash[numPart] = sha256HashString.ComputeHash(partQueue.Dequeue()).ToString();

                numPart++;
            }
        }
        catch (Exception e)
        {
            Console.WriteLine("Error: {0}\nStackTrace: {1}", fe.Message, fe.StackTrace);
        }
    }

    private void hashWrite()
    {
        try
        {
            Console.WriteLine("\nResult:\n");                
            for (int i = 0; i < numParts; i++)
            {
                Console.WriteLine("{0} : {1}", i, hash[i]);
            }
        }
        catch(Exception e)
        {
            Console.WriteLine("Error: {0}\nStackTrace: {1}", fe.Message, fe.StackTrace);
        }
    }

    public void threadsControl()
    {
        try
        {
            Thread readingThread = new Thread(readFromFile);
            Thread calculateThread = new Thread(hashToArray);

            readingThread.Start();
            calculateThread.Start();

            readingThread.Join();
            calculateThread.Join();

            hashWrite();
        }
        catch (Exception e)
        {
            Console.WriteLine("Error: {0}\nStackTrace: {1}", fe.Message, fe.StackTrace);
        }
    }
}

Answer 1

You should read some books about .NET internals before you writing such code. Your understanding of .NET memory model is completely wrong, this is why you getting such error. OutOfMemoryException occurs very rarely, if you care about your resourses, especially if you are dealing with arrays.

You should know that in .NET runtime there are two heaps for reference objects, basic one, and Large Objects Heap, and the most important difference between them is that LOH doesn't being compacted even after garbage collection.

You should know that all the arrays, even small ones, are going to the LOH, and the memory is being consumed very quickly. Also you should know that this line:

part = null;

doesn't dispose memory immidiately. Even worse, this line doesn't do anything at all, because you still have a reference to the part of the file you've read in the queue. This is why your memory goes out. You can try to fix this by calling the GC after each hash computing, but this is highly not recommended solution.

You should rewrite your algorithm (which is very simple case of the Producer/Consumer pattern) without storing whole file contents in memory simultaneously. This is quite easy - simply move out your part variable to the static field, and read the next file part into it. Introduce the EventWaitHandle (or one of it's child classes) in your code instead of queue, and simply compute the next hash right after you've read the next part of file.

I recommend you to start from the basics in threading in C# by reading the great series of Joe Albahari, and only after that try to implement such solutions. Good luck with your projects.