4

I have a method that needs to process a user-supplied file and if the file is complex enough I may encounter an OutOfMemoryException. In these cases I'm fine with the method failing however I would like the application to remain usable, the problem is that after getting an OutOfMemoryException I can't do anything since every course of action consumes memory.

I thought of putting aside some memory which I can free once the exception is throw so the rest of the application can carry on but it seems that the optimizer gets rid of this allocation.

public void DoStuff(string param)
{
    try
    {
#pragma warning disable 219
        var waste = new byte[1024 * 1024 * 100]; // set aside 100 MB
#pragma warning restore 219

        DoStuffImpl(param);
    }
    catch (OutOfMemoryException)
    {
        GC.Collect(); // Now `waste` is collectable, prompt the GC to collect it
        throw; // re-throw OOM for treatment further up
    }
}

Long story short my questions are:

  1. Is there a better way to do what I'm attempting?
  2. If not, is there a good reason why this is a bad idea?
  3. Assuming this idea is the way to go, how do I force the JIT to not optimize away my wasted memory?
Motti
  • 110,860
  • 49
  • 189
  • 262
  • Make your program 64bit, and add this line to your app.config: http://msdn.microsoft.com/en-us/library/hh285054(v=vs.110).aspx – It'sNotALie. Jun 09 '13 at 11:15
  • Or alternatively check the file size. – It'sNotALie. Jun 09 '13 at 11:16
  • @newStackExchangeInstance I don't want to put arbitrary limits on the file size and anyway the amount of memory consumed isn't strictly correlated to file size. As for 64 bit, this is currently not an option since much of the other parts of the application are C++ and thus changing to 64 bit has a lot of repercussions. – Motti Jun 09 '13 at 11:26
  • Have you tried with a [MemoryMappedFile](https://msdn.microsoft.com/en-us/library/system.io.memorymappedfiles%28v=vs.110%29.aspx) ? The inside logic may help to avoid waste of memory on error or insuffisant space. – JoeBilly Mar 15 '15 at 17:55

2 Answers2

2

I wouldn't rethrow an OOM exception, if the memory is already cleaned up.

Otherwise the outer code won't know that, and will think that it needs to free some more memory, whereas it actually doesn't need to.

So I'd opt for throwing a custom exception.

Vlad
  • 35,022
  • 6
  • 77
  • 199
1

If it's a user supplied file, you could check the size of the file before processing it:

FileInfo f = new FileInfo(fileName);
long s1 = f.Length;

This SO thread explains the memory limits of a 32-Bit .NET application. Allocating more than 1,000 MB of memory in 32-bit .NET process

Knowing the size of the file and the amount of memory the application is already consuming can be your check if you need to clean up any resources before attempting to process, or if you'll be able to process at all.

Community
  • 1
  • 1
Shar1er80
  • 9,001
  • 2
  • 20
  • 29
  • Thanks but the complexity of the file does not necessarily correlate with its size. – Motti Mar 15 '15 at 15:39
  • Could you elaborate? You make it sound like a 100MB file could be so complex that, an application would consume X times the size of the file. – Shar1er80 Mar 15 '15 at 15:58
  • Yes that was the case, in the end we did put a cap on file complexity (not size). Thanks for your answer but I'm no longer involved with the project in question – Motti Mar 15 '15 at 19:09
  • A final note to consider... If you're using the contents of a file to instantiate custom classes, I'd suggest making each field to be the data type that holds just above the maximum value needed. For example, if a field is going to hold a value < 255, then you'd only want to use a `byte` instead of an `int` or `short`. – Shar1er80 Mar 15 '15 at 19:22