8

I have problems with a ASP .NET Core 2.1 application running in Windows that increases its memory consumption until finally crashing and requiring to kill the .NET Core Host process. I suspected that the cause could be a synchronization task run in the background once per hour, and I have confirmed that disabling it solves the problem.

I've been profiling this synchronization task with VisualStudio 2019 Diagnostic Tools and I've found a behavior that I don't understand:

enter image description here

As you can see, I have taken 3 snapshots:

  1. At the start of the synchronization method
  2. At the end of the synchronization method
  3. A bit later, once we have existed the scope

In the snapshot table I see a behavior that seem logical to me: the heap size grows considerably during the task (2) and is reduced almost to the initial size (1) when the scope is exited (3). However, the "Process Memory" chart shows a different story: the memory consumption increase is there, but it never goes down. I have launched the application with dotnet run in Release mode and I see the same behavior when looking at the memory used by the .NET Core Host process.

I have two questions:

  1. Why this divergence between the Heap Size and the Process Memory? Shouldn't they be closely related?
  2. Could this be the cause of my webapp crashing? It seems so, but the memory consumption increase should be temporal, not permanent up to the point of crashing it. How could I fix it?

Remark: I have reproduced the same behavior with a simple .NET Core Console App (2.1 and 2.2) with no dependencies, so this is not linked to the ASP part or any other library:

internal class Program
{
    private static void Main()
    {
        new Whatever().AllocateSomeStrings();
        // Snapshot3
        Console.ReadKey();
    }
}

public class Whatever
{
    public void AllocateSomeStrings()
    {
        // Snapshot1
        List<string> numbers = Enumerable.Range(0, 50000).Select(n => n.ToString()).ToList();
        // Snapshot2
    }
}
Tao Gómez Gil
  • 2,228
  • 20
  • 36
  • _“I have reproduced the same behavior with a simple .NET Core 2.1 Console App with no dependencies”_ – Can you share that one? Also, can you verify whether this problem still appears on 2.2 or 3.0? While 2.1 remains supported as an LTS release, it would be interesting to know whether this is still a current issue. – poke Sep 27 '19 at 07:35
  • So when it finally crashes, is it an `OutOfMemoryException` for sure? Have you observed that exception? – Matthew Watson Sep 27 '19 at 07:39
  • 1
    Are you running this on Linux? Server GC usually only releases memory if the system is under memory pressure, which *never* happens on Linux - instead, when memory runs out, the process that has the most memory is killed. – Luaan Sep 27 '19 at 07:41
  • Answers to your questions: I just added the simple Console App code to my question. I've tried with 2.1 and 2.2, same result, not yet with 3.0. I'm running on Windows. I've actually seen the `OutOfMemoryException` – Tao Gómez Gil Sep 27 '19 at 07:56
  • @Luaan, I'm not sure to get the nuance, maybe this will help me understand what's going on: if I see in the snapshot that the Heap Size was reduced, doesn't it mean that the GC released that memory? – Tao Gómez Gil Sep 27 '19 at 07:58
  • @TaoGómezGil It means the heap is smaller. That doesn't necessarily mean the memory is released (as you can see in the diagnostic output) and available to other applications. But that shouldn't mean you get an `OutOfMemoryException`, of course, it just means that the behaviour you observed is not unexpected - that's exactly how a server GC is supposed to behave. The real problem in your actual ASP.NET application might be related to something like heap fragmentation - a memory profiler would be helpful to analyze that. – Luaan Sep 27 '19 at 08:21
  • Hello, did you find a solution to this memory issue in the end? – Konstantinos Papakonstantinou Sep 19 '21 at 18:53

1 Answers1

5

To answer my own questions:

  1. The heap size increases when objects are created, and is reduced when they are GCed. The process memory increases when the heap size increases but it does not necessarily decrease when the heap size decreases. That memory can stay assigned to the process unless some other processes need it (the available memory for the machine goes low), forcing it to be released. This process is not handled by the GC, which operates at the process level. See also: When is memory, allocated by .NET process, released back to Windows
  2. I found that the root problem was a memory leak in the code, a while loop poorly written, fixing that fixed the crashing app.
Tao Gómez Gil
  • 2,228
  • 20
  • 36