List> hitting memory limits

Question

When I open large audio files, I get an out of memory error.

I know this method isn't common, so I thought I have to ask a real boffin.

I have the following happening:

public static List<List<float>> int_filenamewavedataRight = new List<List<float>>();

From there I open up an audio file and load the audio level values in this array, so I can view them correctly through the naudio library.

I clear the arrays like so for a new file:

int_filenamewavedataRight.Clear();

int_filenamewavedataRight.Add(new List<float>());

Then, I load all of the values in to memory for speedy display of the waveforms:

waveStream.Position = 0;
int bytesRead; 
byte[] waveData = new byte[bytesPerSample];
waveStream.Position = 0; // startPosition + (e.ClipRectangle.Left * bytesPerSample * samplesPerPixel);

int samples = (int)(waveStream.Length / bytesPerSample);

wavepeakloaded = 0;
int OldPrecentVal = 0;

for (int x = 0; x < samples; x++)
{
     
    short high = 0;

    bytesRead = waveStream.Read(waveData, 0, bytesPerSample);
    if (bytesRead == 0)
        break;


    for (int n = 0; n < bytesRead; n += 2)
    {
        short sample = BitConverter.ToInt16(waveData, n); 
        if (sample > high) high = sample;
    }

    float highPercent2 = (float)Math.Round(((((float)high) - short.MinValue) / ushort.MaxValue), 2);              

    // ERRORING HERE
    // ERRORING HERE

    int_filenamewavedataRight[filename_value].Add((float)Math.Round(highPercent2, 2)); 

    // ERRORING HERE
    // ERRORING HERE

}

Small audio files, of a typical song with 5 mins in length are fine, but longer files, of 25mins or more, create an exception when a count of 67108864 occurs, and I then get an "Exception of type 'System.OutOfMemoryException' was thrown."

x = {"Exception of type 'System.OutOfMemoryException' was thrown."} 

ex.StackTrace "   at System.Collections.Generic.List`1.set_Capacity(Int32 value)\r\n   at System.Collections.Generic.List`1.EnsureCapacity(Int32 min)\r\n   at System.Collections.Generic.List`1.Add(T item)\r\n   at APP.WaveViewer.LoadWaveToMemory(Int32 filename_value) in  WaveViewer.cs:line 1391"

I'm using a list of a list so that I can address the audio files like a simple array, but as I don't know the initial size of the array, I can specify a size initially.

I'm also pre-loading the waveform data like this so I can have playback, zoom and pan the audio file at the same time.

Is this easily fixable, or should I find a different way of doing this, such as writing these peak volume values to a temporary file, rather than keeping them in memory, or is there a better way?

I've looked this up in various places on the net, such as here however, it seems like a rare thing to be doing this.

Thanks.

Answer 1

The simple answer is it was running in 32bit (x86) which doesn't have enough allocation for 50,000,000+ samples.

Instead, changing the program to x64 has solved that specific problem.

That's what I like about SO where you can pool resources from so many people and learn as you ask questions.