.net memory is not reclaimed

Question

I have a 'service' (owin/webapi + odp.net managed) that runs with the 'server gc mode' on x64 machine. After executing test suite memory usage was around 2Gb. I 'generated' memory pressure on that machine (another program that consumed all available memory and more). I was expecting that windows will force .net to release some memory. However it did not. Here is an output from windbg:

0:018> !dumpheap -stat
Statistics:
              MT    Count    TotalSize Class Name
00007ffd10445b90        1           24 System.Collections.Generic.GenericEqualityComparer`1[[System.UInt64, mscorlib]]
...
00007ffd0f904918   100864     11616976 System.Object[]
0000005b19face20       84   1810674044      Free
Total 720544 objects
Fragmented blocks larger than 0.5 MB:
            Addr     Size      Followed by
0000005b1c3c8800  425.6MB 0000005b36d56c50 System.Func`2[[System.IAsyncResult, mscorlib],[System.Net.HttpListenerContext, System]]
0000005c1c3fcf00   28.2MB 0000005c1e02be58 System.Reflection.Emit.MethodBuilder
0000005c1e02e5b0  268.0MB 0000005c2ec313d8 System.Threading.OverlappedData
0000005c2ec31678  142.3MB 0000005c37a830a0 System.Func`2[[System.IAsyncResult, mscorlib],[System.Net.HttpListenerContext, System]]
0000005d1c541aa0  404.8MB 0000005d35a1a958 System.Func`2[[System.IAsyncResult, mscorlib],[System.Net.HttpListenerContext, System]]
0000005d35a1e5a8    3.8MB 0000005d35df0c40 System.Func`2[[System.IAsyncResult, mscorlib],[System.Net.HttpListenerContext, System]]
0000005e1cfe7128  444.1MB 0000005e38c0c3f0 System.Byte[]
0000005e38c8cfb8    4.5MB 0000005e39110988 System.Byte[]
0000005e39111bf8    1.2MB 0000005e3923a570 System.Byte[]
0000005e3923b7e0    0.7MB 0000005e392ed7d0 System.Byte[]
0000005e392eea40    1.1MB 0000005e39401910 System.Byte[]

0:018> !heapstat
Heap             Gen0         Gen1         Gen2          LOH
Heap0       455256040           24           24      3713672
Heap1       464550872           24      4327776       727456
Heap2       428541352     10344768        12616           24
Heap3       474250128           24     21350456           24
Total      1822598392     10344840     25690872      4441176

Free space:                                                 Percentage
Heap0       446429064            0            0      1819552SOH: 98% LOH: 48%
Heap1       459823968            0          144          152SOH: 98% LOH:  0%
Heap2       428520936        34904           24           24SOH: 97% LOH:100%
Heap3       474044952            0          336           24SOH: 95% LOH:100%
Total      1808818920        34904          504      1819752

So there is only around 48Mb really used and 1.8Gb free, most of memory is not in LOH but in Gen 0 heap, but .net would not give that memory back. And meanwhile system was starving for memory. For example IIS failed to start because there were not enough memory.

Why memory was not reclaimed by windows?

16, but there are other services. I wrote that I simulated high memory pressure by running another tool that occupied all memory available — koruyucu, Aug 10 '15 at 14:16
Please read the question first. Its not IIS, not SQL. Oracle is running on a linux server in the data center located in another city, if you wonder. I have dedicated server that runs several services and one of them consumes all memory and does not give it up. I could run my tests on bigger datasets and occupy whole 16Gb. The problem is there are only 50Mb used and .net captures 2Gb and does not give it back to OS. — koruyucu, Aug 10 '15 at 14:31
I see nothing wrong with CLR reserving the memory. The testing setup (one .NET app with another memory hungry app) is similar to running both IIS and SQL Server on the same machine, which if you search around is a common wrong approach that ends with memory issues. — Lex Li, Aug 10 '15 at 14:34
Almost a gigabyte worth of `Func` delegate objects is not normal. Pretty hard to imagine code that could do that. The typical object pinning that goes along with async I/O is pretty likely to blow up the GC. — Hans Passant, Aug 10 '15 at 16:05
Service is reading big results from oracle. 10 columns and 800K rows. So that much memory is actually used when reading response of a db (there is extremely inefficient implementation to store results). After response is sent gc is triggered and reclaims memory. As service works in the 'server' mode according to [official documentation](https://msdn.microsoft.com/en-us/library/ee787088%28v=vs.110%29.aspx) then size of 'ephemeral segment' is 2Gb for the server service is running. So I suppose .net would not give a part of that segment to OS. Does it sound anything like true? — koruyucu, Aug 11 '15 at 10:11
I guess I've found the problem: http://stackoverflow.com/questions/10686346/how-to-stop-asynchronous-tcp-net-code-from-using-up-an-entire-systems-resource — koruyucu, Aug 26 '15 at 11:18
and don't you put those big data's into memory? on a singleton? singleton objects won't free up until you close application, for example you wanna reference some data around your app, and you fetch them all before a request come. it won't despose, and your application won't loss link to it, so even if you call GC.Collect, it wont be collected, i do not understand these analyse sheets, but i hope this help you — Hassan Faghihi, Aug 30 '15 at 11:26

score 2 · Answer 1 · edited May 23 '17 at 11:50

Well windows cannot "force" .Net to free memory. While I am no expert in the .Net runtime my experience has been that the .net framework will hold onto memory once your application starts to consume a lot of it. It will slowly give it back as it see's fit. From windows perspective it doesn't really know what .Net will do with it but it .Net has claimed that memory from the OS and so until it (.net) gives it back windows cannot give that memory to another application. There are several 400+mb chunks of System.Byte[] that are being held onto, which could be indicative of a memory leak. From the OS's perspective the Framework has requested the memory and even used that memory so until it is explicitly (from the framework not you) given back windows cannot do anything about it. I would profile your application very carefully because generally the reason .Net isn't releasing memory is because you have a memory leak. Using the memory analysis tools in Visual Studio should show you if you are leaving memory "Pinned" or just holding on to it. Really bad culprites are often static class variables/events.

SO link: .NET Free memory usage (how to prevent overallocation / release memory to the OS)

score 0 · Answer 2 · answered Sep 01 '15 at 06:37

Since, windows cannot compell the app to release objects.You need to find a way to define Max memory usage for the application & when the max limit is reached call the garbage collector manually via GC.Collect().

While the application is runnning,Try to get the Process & set the maxLimit,similar to this

.net memory is not reclaimed

2 Answers2