Cleaning up large Objects from IIS Memory

Question

I have an application that stores DataTables in an in memory cache (AppFabric). These DataTables can be quite large. Once there is a lot of traffic from our app (MVC site) the memory usage of IIS goes through the roof very quickly.

Ideally we want to be able to release the memory that these DataTables consume once requested from the Cache.

The code of the Controller is something along the lines of

Using (DataTable dt = DataTable)
{
    DataTable dt = Cache.GetObject(objectID);

    //perform some manipulation on Data table
    DataTable dtSmaller = dt.Select("Select top 1...");

   dt.Dispose();
}   
    //return from controller
return dtSmaller;

Once this controller is hit many times the W3WP.exe process uses masses of memory until eventually going out of memory. What is happening is that a DataTable comes from the Cache, it's queried to reduce the size of the output data. I then dispose the original DataTable.

I was looking for a way of releasing the memory consumed by the DataTable without the depending on IIS Garbage Collection

You are not responsible for releasing memory but the garbage collector. Why don't you use a database? — Tim Schmelter, Sep 02 '14 at 12:23
I still don't understand what consumes so much memory. You cache seems to contain a large `DataTable`. But that size is fix and should not be a problem. So are a great number of smaller tables causing this memory issue? I cannot believe that. What is `dt.Select("Select top 1...");` actually doing since that is not valid syntax? You could use LINQ: `dt.AsEnumerable().Take(10)` which does not need to create a new `DataRow[]` always. So don't return `DataTable` but `IEnumerable`. — Tim Schmelter, Sep 02 '14 at 12:31
The select is only for demonstrating. Basically it runs a query on the data table and returns a smaller portion of Data. The Cache is a different application (MS App Fabric). Once the IIS process reads the DataTable from the cache it creates a local copy. Each request to the controller appears to be creating more and more local copies eventually using all available memory. — keitn, Sep 02 '14 at 12:38

score 0 · Accepted Answer · answered Sep 02 '14 at 13:16

0

You can force a full Garbage Collection cycle by calling GC.GetTotalMemory(true)

More info on the method: GC.GetTotalMemory

answered Sep 02 '14 at 13:16

toadflakz

7,764
1
27
40

I think through a mix of forcing GC and using WeakReferences I should be able to achieve what I want. – keitn Sep 03 '14 at 08:15

Cleaning up large Objects from IIS Memory

1 Answers1

Linked