Why is the memory still occupied after ‘list = null’ or 'list.clear()'

Question

I have a function which get sqls from a .sql file(about 400k).Then execute them. But I don't know why the memory used has increased so much(250MB for first execute).I tried 'list = null' and 'list.clear()'.But the memory used cannot be reduced. And the memory used will increase about 100MB after I execute the function.This code is written by resigned personnel.

I want to know how to solve it or make memory won't increase after each execution.

Sorry for my poor English.memory used in visualstudio

Code here

public static void UpgradeDatabase(DbHelper db,string deployPath)
    {
        try
        {
            string sqlPath = Path.Combine(deployPath, "Source", "UpdateDataBase.sql");
            if (!System.IO.File.Exists(sqlPath))
            {
                throw new Exception("");
            }
            List<string> sqlList = GetSqlList(sqlPath, db.Database); //memory used will increase after this
            if (sqlList == null)
            {
                throw new Exception("");
            }
            db.ExecuteSqlList(sqlList);
            sqlList = null;
        }
        catch (Exception ex)
        {
            //
        }
    }

another function

private static List<string> GetSqlList(string sqlPath, string database)
    {
        var sqllist = new List<string>();
        try
        {
            var fileInfo = new FileInfo(sqlPath);
            string commandText = "";
            string varLine = "";
            var reader = new StreamReader(sqlPath, Encoding.GetEncoding("gb2312"));
            while (reader.Peek() > -1)
            {
                varLine = reader.ReadLine();
                varLine = varLine.Replace("[kms]", "[" + database + "]");
                if (varLine == "")
                {
                    continue;
                }
                if (varLine.Trim() != "GO" && varLine.Trim() != "go")
                {
                    commandText += varLine;
                    commandText += "\r\n";
                }
                else
                {
                    sqllist.Add(commandText);
                    commandText = "";
                }
            }
            reader.Close();
            if (!string.IsNullOrEmpty(commandText))
            {
                sqllist.Add(commandText);
            }
            return sqllist;
        }
        catch (Exception ex)
        {
            return null;
        }
    }

When you put `list.Clear();` or `list = null` Grabage Collector (GC) doesn't necessary start collecting at once; technically, you can force garbage collection by `GC.Collect(2);` but it's often not a good practice. — Dmitry Bychenko, Aug 17 '22 at 08:32
How come this was closed as a duplicate of "How do I sort a list of dictionaries by a value of the dictionary"? — Orion, Aug 17 '22 at 08:35
I've tried GC.Collect(2), it seems didnt work for first execution. but memory wont increase after each execution. sorry ,I didnt find how to reply a comment — remilialover, Aug 17 '22 at 08:40
@Orion I'm very confused about that myself! I think I must have messed up in the close dialog, though I did paste the URL to the right question. I've fixed it now. Sorry for the confusion. — ProgrammingLlama, Aug 17 '22 at 08:40
OP: I suggest you write `using (var reader = new StreamReader(sqlPath, Encoding.GetEncoding("gb2312"));) {` (enclosing the reader in a using block) and place the corresponding `}` below `reader.Close();` (which you can remove if you make this change). This won't really change the memory usage though. Is the memory usage really a problem? — ProgrammingLlama, Aug 17 '22 at 08:42
@DiplomacyNotWar yes,the most memory are occupied by this problem on the server.It made other applications canont work Normally. — remilialover, Aug 17 '22 at 08:49
How do you measure memory usage? Even when GC collects garbage, the runtime is not necessarily going to give back the memory to OS. It may keep it for future usage, and AFAIK the runtime often does exactly that. — freakish, Aug 26 '22 at 18:58

Dmitry Bychenko · Accepted Answer · 2022-08-17T09:01:14.820

You create too many strings and throw them away (strings are immutable and can't be changed): just have a look at commandText += in the loop. You can use StringBuilder instead:

using System.IO;
using System.Linq;

...

private static List<string> GetSqlList(string sqlPath, string database) {
  var sqllist = new List<string>();

  StringBuilder sb = new StringBuilder();

  // Linq : let's prepare data to work with 
  var commands = File
     .ReadLines(sqlPath, Encoding.GetEncoding("gb2312"))
     .Where(line => !string.IsNullOrWhiteSpace(line))
     .Select(line => line.Replace("[kms]", $"[{database}]"));

  foreach (string line in commands) {
    if ("go".Equals(line.Trim(), StringComparison.OrdinalIgnoreCase)) {
      sqllist.Add(sb.ToString());
      sb.Clear();
    }  
    else  
      sb.AppendLine(line); 
  }

  if (sb.Length > 0)
    sqllist.Add(sb.ToString());

  return sqllist; 
}

Please note, that once you put list.Clear(); or list = null; you just allow Garbage Collector to collect unused list items; it does't mean that Garbage Collector must start collecting the items. Technically, you can force garbage collection by GC.Collect(2); but it's not a good practice in many a case.

It works.Thanks a lot.And my whole function(it's webapi) will make memory usage to increase about 5MB.Is it will be collected by GC or IIS? Or unlimited increase by each execution? — remilialover, Aug 17 '22 at 09:23
@remilialover: as for current methods, I can't see any leakage, so the garbage will be collected and memory usage should be stable. As for other methods, classes etc. (esp. `static` ones) if you see constantly *increasing* memory usage (not zigzag one) you can well have a leakage — Dmitry Bychenko, Aug 17 '22 at 09:59

Why is the memory still occupied after ‘list = null’ or 'list.clear()'

1 Answers1