Out of memory exception when not using all the memory/limits

Question

We have an issue here where we can have some OutOfMemoryException.

We will check how we can reduce the memory usage, but my question is why I get it at this point.

According to the Memory profiler, and the windows task manager, the application weights only 400MB.

For what I understood(confirmed here), for 32bits applications, the limitation should be around 2GB. My computer has 16GB of ram, and there is plenty of ram available(more than 4GB).

So why do I get this error now?

My question is not about why my application has its memory growing, but more to understand why it's already happening now. I've the feeling that this limit is not a fixed one, but I can't find any reference on this.

The call stack if it helps:

System.OutOfMemoryException: Out of memory.
   at System.Drawing.Graphics.CheckErrorStatus(Int32 status)
   at System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   at Nevron.GraphicsCore.NBitmapGdiRenderSurface.Paint(Object sender, PaintEventArgs e, l1ll11Il1 contentPainter)
   at Nevron.Chart.WinForm.NControlView.Paint(Object sender, PaintEventArgs e)
   at Nevron.Chart.WinForm.NChartControl.OnPaint(PaintEventArgs e)
   at System.Windows.Forms.Control.PaintWithErrorHandling(PaintEventArgs e, Int16 layer)
   at System.Windows.Forms.Control.WmPaint(Message& m)
   at System.Windows.Forms.Control.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Edit I got the same exception with a totally different stack trace:

System.ComponentModel.Win32Exception (0x80004005): Not enough storage is available to process this command
   at DevExpress.Utils.Drawing.XtraBufferedGraphicsContext.CreateCompatibleDIB(IntPtr hdc, IntPtr hpal, Int32 ulWidth, Int32 ulHeight, IntPtr& ppvBits)
   at DevExpress.Utils.Drawing.XtraBufferedGraphicsContext.CreateBuffer(IntPtr src, Int32 offsetX, Int32 offsetY, Int32 width, Int32 height)
   at DevExpress.Utils.Drawing.XtraBufferedGraphicsContext.AllocBuffer(Graphics targetGraphics, IntPtr targetDC, Rectangle targetRectangle)
   at DevExpress.Utils.Drawing.XtraBufferedGraphicsContext.AllocBufferInTempManager(Graphics targetGraphics, IntPtr targetDC, Rectangle targetRectangle)
   at DevExpress.Utils.Drawing.XtraBufferedGraphicsContext.Allocate(Graphics targetGraphics, IntPtr targetDC, Rectangle targetRectangle)
   at DevExpress.Utils.Drawing.XtraBufferedGraphicsContext.Allocate(Graphics targetGraphics, Rectangle targetRectangle)
   at DevExpress.XtraBars.Docking2010.Views.BaseViewPainter.Draw(GraphicsCache cache, Rectangle clip)
   at DevExpress.XtraBars.Docking2010.Views.BaseView.Draw(GraphicsCache cache, Rectangle clip)
   at DevExpress.XtraBars.Docking2010.DocumentManager.PaintCore(Graphics g, Rectangle bounds)
   at DevExpress.XtraBars.Docking2010.DocumentManager.DevExpress.XtraBars.Docking2010.IDocumentsHostOwner.Paint(Graphics g)
   at DevExpress.XtraBars.Docking2010.DocumentsHost.OnPaint(Graphics g)
   at DevExpress.XtraBars.Docking2010.DocumentsHost.DoPaint(Message& m)
   at DevExpress.XtraBars.Docking2010.DocumentsHost.WndProc(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
   at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Can you post the code that renders the image or a cut down version of it? That may be the issue. See http://stackoverflow.com/q/16055667/1239433 — NeddySpaghetti, Oct 14 '15 at 11:15
Unfortunately, I don't have it, we use Nevron to draw a plot(actually, to draw 20 plots in parallels), and we just add a value to a series, that is replicated here.). — J4N, Oct 14 '15 at 11:27
how do you reproduce the problem? Do you open and close forms and windows? — NeddySpaghetti, Oct 14 '15 at 11:38
No, I just let the application open, drawing all the data it receives(a lot of data every seconds) — J4N, Oct 14 '15 at 11:46

score 5 · Answer 1 · answered Oct 14 '15 at 10:36

5

One possibility is that your large object heap has become fragmented. I.e. it has plenty of space but no space large enough to satisfy an allocation of a large object.

The LOH is not usually compacted although there seem to be a way to do a one off compaction now.

If that is indeed the issue, one way to avoid LOH fragmentation is through the use of pools of large objects that just get returned to the pool when you are done with them rather than letting the GC deal with them.

answered Oct 14 '15 at 10:36

NeddySpaghetti

13,187
5
32
61

Okay, how can I confirm this is the issue or not? – J4N Oct 14 '15 at 10:49
Try using the trial version of ANTS Profiler, it has a feature to detect LOH fragmentation https://www.simple-talk.com/dotnet/.net-framework/large-object-heap-compaction-should-you-use-it/ – NeddySpaghetti Oct 14 '15 at 10:51
I've a license for that tool ;) I see only in "Session overview" -> "Large Object Heap size", where I've ~2MB of used space and 12MB of unused object space. Is that an issue? Is there something else I can check on ANTS for this? – J4N Oct 14 '15 at 10:53
Another tool that I really like for memory leaks is http://memprofiler.com/, but not sure if it can do LOH fragmentation detection. – NeddySpaghetti Oct 14 '15 at 10:53
Are you able to get a trace for the exception? Then you can see what allocation throws the error. – NeddySpaghetti Oct 14 '15 at 10:55
We store them automatically on the disk, I added it to the question – J4N Oct 14 '15 at 11:08
Regarding the usage of ANTS for LOH fragmentation. I had sometime a warning of ANTS, but, I don't feel it was justified since the "usable space" of the LOH was still a very high percentage – J4N Oct 15 '15 at 07:26
@J4N I think this is likely an issue with how the particular charting framework is used. I'd double check with the vendor documentation to see if you are using it correctly, or if it has a bug. – NeddySpaghetti Oct 15 '15 at 12:12
Well I've already contacted them, but I'm pretty sure they will ask for some sample, which I'm unable to reproduce :(. At least the LOH is mostly used by them. Thank you anyway – J4N Oct 15 '15 at 12:31

dandan78 · Answer 2 · 2015-10-14T11:17:17.373

4

It is possible that you are seeing the results of memory fragmentation. This occurs when you have, say, 2GB free in total, but your biggest continguous chunk of memory is much smaller than that, for instance, 200MB. Then you try to allocate a 400MB chunk and get an out of memory exception.

Also, if you are working with GDI+, it has a nasty habit of throwing out of memory exceptions in all kinds of situations, most of which have nothing to do with memory.

Having just looked at your trace, it could easily be some of that GDI+ wierdness I mentioned. In the past I've had this happen with file permissions, like when loading a bitmap. That's what I would check if I were you. Not specifically file permissions but situations where GDI+ is known to spit out OOM exceptions.

edited Oct 14 '15 at 11:17

answered Oct 14 '15 at 10:35

dandan78

13,328
13
64
78

We are handling a lot of data, so we have to do a lot of create/destruct of some classes, how can I be sure this is because of the memory fragmentation? What would be the direction to take to solve this ? – J4N Oct 14 '15 at 10:52
I can't really add anything to what Ned has already posted. You could Google for .Net memory profiler and see what that produces. – dandan78 Oct 14 '15 at 10:55
It makes somethings like two day that I'm running a memory profiler :(. I don't see the LOH growing if that's what I'm supposed to look for. – J4N Oct 14 '15 at 11:09
I turned ON the hardware acceleration on Nevron, which doesn't use anymore GDI to render the charts, and I now have another `OutOfMemory` with a different callstack(DevExpress drawing a bar, not sure this is using GDI) – J4N Oct 14 '15 at 11:30
If it's GDI+ issue, what could I do? Because for now, it seems that always when calling method that at the end call some GDI methods, but I don't see what I can do for this – J4N Oct 14 '15 at 12:34
I've done some additionals checks, I've downloaded GDIView, which is displaying me the number of GDI elements for every windows application I've, but nothing is growing inexplicabely – J4N Oct 15 '15 at 07:23

Out of memory exception when not using all the memory/limits

2 Answers2

Linked