0

We are seeing OutOfMemory errors on our production server. The application is a 64-bit application (.Net Core running on top of the .Net Framework 4.8). My understanding is that a 64-bit .Net application should be able to access up to 8 TB of memory, constrained by the available physical memory and the pagefile size. The memory usage within IIS is approximately 2.8 GB when the error happens. The server has 64 GB of memory. The PageFile size on the server is a little over 9 GB (managed by Windows). Maximum ram used on server: 14 GB. How can we be running out of memory in this situation?

JRS
  • 569
  • 9
  • 26
  • 1
    GDI+ gives an "Out of memory" error for things like an image format it doesn't understand, if that's applicable. – Andrew Morton Jun 24 '22 at 16:35
  • There is no image processing going on. – JRS Jun 24 '22 at 16:38
  • Can you narrow down where the error occurs to an exact line of code? – Andrew Morton Jun 24 '22 at 16:41
  • Pretty much, yes. It is coming from an Entity Framework query that is pulling all records in table. The table has between 70,000 and 100,000 records - no images in any field. (No joins to any other tables in the query.) – JRS Jun 24 '22 at 16:51
  • Does [Entity framework large data set, out of memory exception](https://stackoverflow.com/questions/18169859/entity-framework-large-data-set-out-of-memory-exception) help? – Andrew Morton Jun 24 '22 at 16:54
  • I don't think so. That article discusses an EF query pulling 2 million records. 100K is a long way from that. Still, I have re-written the query to pull only a subset of the fields of the table, and this may help. But regardless, I don't understand the OOM error - we have plenty of memory on the server, so why are we getting the error? – JRS Jun 24 '22 at 16:59
  • At a guess, the 2 million records might have been very small records. At another guess, maybe some individual object has reached 2 GB in size. Hopefully, someone with more knowledge will be along to answer. – Andrew Morton Jun 24 '22 at 17:04
  • 1. .NET Core on .NET Framework is dead, as all releases supporting that feature are end of life. 2. OutOfMemoryException in a 64 bit application is more likely to be caused by memory fragmentation, so the actual memory layout and the exception call stack are critical for troubleshooting and you need to capture full memory dumps for further investigation. 3. The bar to work on such tasks is rather high, so if you are not familiar please hire an experienced professional or simply contact Microsoft support for assistance. – Lex Li Jun 24 '22 at 17:13
  • @LexLi Just to clarify, ASP.NET Core 2.1 on .NET Framework isn't dead at this time: https://dotnet.microsoft.com/en-us/platform/support/policy/aspnetcore-2.1 – kapsiR Jun 24 '22 at 17:21
  • @JRS Can you provide a little bit more information? (e.g. stack trace, do you surely know where it happens?) – kapsiR Jun 24 '22 at 17:23
  • @kapsiR Thanks for adding that, but such statements/support policies (as "Tools") only covers security patches. The ecosystem has already moved away to .NET 5/6 and breakage is unavoidable. – Lex Li Jun 24 '22 at 17:33
  • As long as you can't provide anything that helps from our point of view, I can recommend you reading this: [Troubleshooting System.OutOfMemoryExceptions in ASP.NET](https://learn.microsoft.com/en-us/troubleshoot/developer/webapps/aspnet/health-diagnostic-performance/troubleshoot-outofmemoryexception) – kapsiR Jun 24 '22 at 17:47
  • @kapsiR The error is coming from an Entity Framework query that is pulling all records in table. The table has between 70,000 and 100,000 records - no images in any field. (No joins to any other tables in the query.) – JRS Jun 24 '22 at 18:48
  • @JRS When does the exception happen? (when the query is written into a big array for example?) – kapsiR Jun 24 '22 at 19:27
  • Did you try enable [gcAllowVeryLargeObjects](https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element)? In case of EF, did you try calling it with `.AsNoTracking()` to reduce the memory? Btw. the [OOM documentation](https://learn.microsoft.com/en-us/dotnet/api/system.outofmemoryexception) is pretty good too! – kapsiR Jun 24 '22 at 19:28
  • @kapsiR I did re-write the method at issue to pull only a subset of the table fields, and to populate a custom object which is not change-tracked. This is working on our test environment and will be applied to production tonight. But still doesn't explain why we get OOM error with plenty of memory - maybe Lex Li is correct. (I did not try enabling gcAllowVeryLargeObjects, but I can try that. Thank you.) – JRS Jun 24 '22 at 20:10
  • @LexLi I understand that ASP.NET Core 2.1 on .NET Framework is not ideal, but this is an older application, and it uses EntityFramework 6.0. This is not supported in .Net Core, which means that moving to .Net Core alone (without the Framework) will require a complete re-write of our data access layer. This is a large application, so that is a substantial obstacle. – JRS Jun 24 '22 at 20:23
  • 1
    EF 6.3 and above supports .NET Core (3.0 and above), https://learn.microsoft.com/en-us/ef/ef6/what-is-new/ So there is a possible migration path for you to get rid of legacy .NET Framework and ASP.NET Core 2.1. With about 4 years of advancement and many issues resolved, .NET 6+EF 6.4 should give you a better ground for this application. For the issue alone thorough analysis is needed to understand what's wrong. You can see what Microsoft says about EF 6 support from "Getting help" https://github.com/dotnet/ef6#getting-help – Lex Li Jun 24 '22 at 22:54

0 Answers0