Why is my .net application hogging RAM?

Question

I have added some .dispose() and .close() calls which seem to have helped a fraction but i'm still seeing the application using 600MB of RAM once all operations are complete. What am i missing?

public partial class Form1 : Form
{
    public Form1()
    {
        InitializeComponent();
    }

    public void CreateCSVFile(DataTable dt, string strFilePath)
    {
        #region Export Grid to CSV
        // Create the CSV file to which grid data will be exported.
        StreamWriter sw = new StreamWriter(strFilePath, false);
        // First we will write the headers.
        //DataTable dt = m_dsProducts.Tables[0];
        int iColCount = dt.Columns.Count;
        for (int i = 0; i < iColCount; i++)
        {
            sw.Write(dt.Columns[i]);
            if (i < iColCount - 1)
            {
                sw.Write(",");
            }
        }
        sw.Write(sw.NewLine);
        // Now write all the rows.
        int current_row = 0;
        foreach (DataRow dr in dt.Rows)
        {
            current_row++;
            if (current_row % 1000 == 0)
            {
                lbl_progress.Text = current_row.ToString() + " of " + dt.Rows.Count.ToString();
                this.Refresh();
                Application.DoEvents();
            }
            for (int i = 0; i < iColCount; i++)
            {
                if (!Convert.IsDBNull(dr[i]))
                {
                    if (dr[i] is string)
                    {
                        sw.Write("\"" + dr[i].ToString().Replace(Environment.NewLine,", ").Replace("\"", "") + "\"");
                    }
                    else
                    {
                        sw.Write(dr[i].ToString());
                    };
                }
                if (i < iColCount - 1)
                {
                    sw.Write("|");
                }
            }
            sw.Write(sw.NewLine);
        }
        dt.Dispose();
        sw.Close();
        #endregion
    }

    private void button1_Click(object sender, EventArgs e)
    {
        button1.Enabled = false;
        string strConn = ConfigurationManager.AppSettings["sql_connection"].ToString();
        SqlConnection conn = new SqlConnection(strConn);
        for (int i = 1; i <= int.Parse(ConfigurationManager.AppSettings["query_count"].ToString()); i++)
        {
            try
            {
                SqlDataAdapter da = new SqlDataAdapter(System.Configuration.ConfigurationManager.AppSettings["sql" + i.ToString()].ToString(), conn);
                DataSet ds = new DataSet();
                da.Fill(ds);
                da.Dispose();
                DataTable dt = ds.Tables[0];
                ds.Dispose();
                textBox1.Text += "\r\n[" + DateTime.Now.ToString("HH:mm:ss") + "] Starting sql" + i.ToString();
                this.Refresh();
                Application.DoEvents();
                CreateCSVFile(dt, System.Configuration.ConfigurationManager.AppSettings["output_path"].ToString() + i.ToString() + ".csv");
                textBox1.Text += "\r\n[" + DateTime.Now.ToString("HH:mm:ss") + "] Finished sql" + i.ToString();
                dt.Dispose();
                this.Refresh();
                Application.DoEvents();
            }
            catch (Exception ex)
            {
                textBox1.Text += "\n" + ex.Message;
                //break;
            }
        }
        textBox1.Text += "\r\n[" + DateTime.Now.ToString("HH:mm:ss") + "] All done";
        button1.Enabled = true;
    }

}

Is that the whole thing? Your program consists entirely of this? — Jack, Apr 30 '12 at 09:06
How many rows/columns are there? Could just be the data table is huge... — Adam Houldsworth, Apr 30 '12 at 09:06
Umm, how big is your data set? Surely that's the primary piece of information. Here's a decent litmus test - how big is the CSV file once written out? — yamen, Apr 30 '12 at 09:08
Closing / disposing doesn't necessarily trigger garbage collection. You can call [`GC.Collect`](http://msdn.microsoft.com/en-us/library/xe0c2357.aspx) if you really do need to do that, but [it's not necessarily recommended](http://stackoverflow.com/questions/233596/best-practice-for-forcing-garbage-collection-in-c-sharp) - do you need to release the memory? — Rup, Apr 30 '12 at 09:09
Oh, misc style: it's safer to use [`using` scopes](http://msdn.microsoft.com/en-us/library/yh598w02.aspx) rather than explicitly calling dispose yourself because it'll still get called if an exception is thrown. Also you could use Environment.NewLine rather than "\r\n" — Rup, Apr 30 '12 at 09:13
How have you measure the 600MB? I am using http://technet.microsoft.com/en-us/sysinternals/bb896653 and there you can exactly see how RAM works. — Nasenbaer, Apr 30 '12 at 09:13
This is the whole thing minus the .config. I didn't see the relevance of the dataset as the RAM is being held after everything has finished. I will try GC.Collect in my next development release and update. Thanks — Lee Tickett, Apr 30 '12 at 09:26

Jirka Hanika · Answer 1 · 2012-04-30T11:08:52.983

Take the following actions to take down those perceived 600 MB to measured 0 bytes:

All your Dispose and Close calls belong to finally blocks to be exception safe; or better yet, learn about the using statement as already advised by a comment. This style problem is at the root of many fatal memory leaks but probably not in your case given how small your sample is.
To measure memory allocated and kept, do a GC.GetTotalMemory(true) before and after the measured block. Do not leave this in production code, unless you understand and are willing to pay the performance cost.
Dispose also the database connection. (This is trivial in terms of client side memory consumption, but not doing so may have various side effects. Connections are an exhaustible resource themselves, they consume memory also on the database server, and stale connections can block some database operations or confuse humans.)
Run the whole measurement cycle in the same process twice and disregard the first set of results. .NET framework may allocate some memory for its own static structures during the dry run. A memory leak is only worth your attention if it keeps growing or has a potentially unlimited size.

Adam Houldsworth · Accepted Answer · 2012-04-30T12:11:07.893

Firstly I'll say that Disposal has nothing to do with garbage collection - disposing something doesn't mark it for GC nor reclaims any managed memory. It is a mechanism by which code can deterministically release resources, usually external or unmanaged.

Pulled off of MSDN:

Garbage collection occurs when one of the following conditions is true:

The system has low physical memory.

The memory that is used by allocated objects on the managed heap surpasses an acceptable threshold. This means that a threshold of
acceptable memory usage has been exceeded on the managed heap. This
threshold is continuously adjusted as the process runs.

The GC.Collect method is called. In almost all cases, you do not have to call this method, because the garbage collector runs continuously. This method is primarily used for unique situations and testing.

These are the conditions in which the GC will run.

I believe you are getting confused between RAM currently being used and the application simply having a large working set (memory the process has been given by the OS, not necessarily used). It may have used 600Mb at its peak, and because nothing else is asking for more space, the RAM associated with your process is left alone. Task Manager is not a good judge of how much RAM a process is actually using, just how much RAM it has been given to use.

You really need to use a proper memory profiler to see how many objects are alive at a given point in time.

Also, anything that implements IDisposable can be used within a using statement:

using (var connection = new SqlConnection(""))
{
    connection.Open();
} // Dispose is called here, even on exception or return within using.

using statements simply compile into try/finally blocks under the hood.

score 1 · Answer 3 · answered Apr 30 '12 at 09:47

It does not do a garbage colection - because it is not needed?

See, 600mb may not be a lot on your computer, and the GC only kicks in when space is needed (i.e. other applications ask for it) or a threshhold is reached.

SO, unless that is a problem, it may simple be a wrong obersvation.

Run a memory profiler to see what the real memory load is (it will force a collection, then see what happensand allows you to analyze what uses the memory and why).

Do NOT (!) run GC.Collect manually unless in VERY rare conditions (basically: you are fired for rdoing so, now argue why I should not fire you). I need the memory is NOT such a condition - GC.Collect has bad side effects (miximg up generational statistics).

Why is my .net application hogging RAM?

3 Answers3