0

Possible Duplicate:
TStringList of objects taking up tons of memory in Delphi XE

I have a program that takes a while to run and uses a fair amount of memory (starts out at about 800 mb, ends at about 1.1 GB).

The problem I'm having is that even though the amount of memory it's using is fine, the amount of memory commited goes over 4GB and gives me an "Out of Memory" error.

Here is a screen shot of Resource Monitor:

A screen shot of Resource Monitor

The main thing that takes up the memory is a TStringList filled with an object that has a bunch of different values and a few stringlists

Does anyone know why it is committing so much more memory than I need, and how can I make my program not do that?

Community
  • 1
  • 1
KingOfKong
  • 279
  • 7
  • 17
  • 6
    Maybe memory leak? – liuyanghejerry Nov 07 '11 at 14:56
  • I'm not sure, the memory (at least working memory) rises steadily as I add objects to the TStringList and it doesn't rise at all if I don't add any objects after doing so initally. – KingOfKong Nov 07 '11 at 15:13
  • Which version of Delphi are you using? Are you using FastMM4 for older version of Delphi? If not - you should - and use its memory leak tracking. – Arnaud Bouchez Nov 07 '11 at 15:27
  • It's not really the same question: The program was using a very large amount of working memory. I cut that usage in less than half so the working memory is fine, but the committed memory is the problem. – KingOfKong Nov 07 '11 at 15:35
  • I'm using Delphi 2007 and I thought it had FastMM4 built in. Unless that's an older version and I should be using a more recent one? – KingOfKong Nov 07 '11 at 15:36
  • @KingOfKong You are right: Delphi 2007 has FastMM4 built in (it was included since Delphi 2006, if I do not have a memory leak). Did you use it to report memory leaks? In all cases, consider using one of the answers from the previous question (which may fit exactly your current question) - and at least accept one! :) – Arnaud Bouchez Nov 07 '11 at 15:54

4 Answers4

1

The commit size is the amount of memory allocated by the program (but is not necessarily in physical memory). As mentioned in the comments, it is quite possible that the program has a memory leak (e.g., it creates objects possibly but does not delete them). So even if the amount of memory in use is "reasonable", it doesn't necessarily represent the amount actually allocated, and that amount in a 32-bit system can't "typically" exceed 4GB. This question discusses leak detection tools that might be useful.

Community
  • 1
  • 1
Mark Wilkins
  • 40,729
  • 5
  • 57
  • 110
1

Your software probably contains a memory leak. Put this little piece of code right after your program's "main" block (.dpr file - press F7 to get there).

   {$WARN SYMBOL_PLATFORM OFF}
   ReportMemoryLeaksOnShutdown := DebugHook <> 0;

So, the end result should look similar to this:

 begin
  {$WARN SYMBOL_PLATFORM OFF}
  ReportMemoryLeaksOnShutdown := DebugHook <> 0;

  Application.Initialize;
  Application.CreateForm(TForm1, Form1);
  Application.Run;
 end.

Then, run your program until you get some big memory usage and simply quit it. Delphi will show a nice window giving you the size of the memory leak as well as some tips about where they might be happening.

Good luck!

ivarec
  • 2,542
  • 2
  • 34
  • 57
  • Wow, thanks, this is really useful. Is there a way to see more info or the full writeup? There are a bunch of Unknown or String unexpected small block leaks and it just lists the sizes of unexpected leaked medium and large blocks without saying what they are – KingOfKong Nov 07 '11 at 17:23
  • This is probably all you will be able to get using tools provided with Delphi. There are some commercial tools that are supposed to give the kind of feedback you are looking for. Try this page for a list of useful tools: http://delphi.about.com/od/toppicks/tp/aatpmemleak.htm. Also, now you KNOW that you got a memory leak there. Its just a matter of finding it. – ivarec Nov 07 '11 at 17:52
  • 2
    @haole You can have a full report of memory leaks (including stack trace at creation), if you download the FastMM4 from sourceforge. – Arnaud Bouchez Nov 07 '11 at 18:06
  • @KingOfKong Perhaps you have a memory leak during *freeing* all your objects - but the question is about having so much memory allocated when *creating* your objects. With this, FastMM4 or other won't help you. And without the actual code of your app, it is difficult to guess what is wrong with it. – Arnaud Bouchez Nov 07 '11 at 18:08
  • @ArnaudBouchez We don't have the code. He might be messing up the creating of those objects, using temporary stuff that gets left behind on each iteration. IMO, it is too much memory above the expected to be anything else than a brute memory leak. But, as you pointed out, without the code it's just a guess. Thanks for the tip about FastMM4. – ivarec Nov 07 '11 at 18:23
0

You didn't showed code, but this effect appears to be the result of a memory leak.
So, this TStringList (I'll call it parent) have objects associated, which have its own child TStringLists (which I suppose have only strings).

The easier way to do this memory management is letting the TStringList do it for you:

  • If in the Delphi version you have TStringList has the OwnsObjects property, set it to true. With this, every call to Delete and Clear methods will call the destructor of the objects associated with the parent list - and this will work even if cll parent.Free. If it doesn't implement, you'll have to do it yourself - overriding the Delete, Clear and Destroy methods of TStringList in a descendant class.
  • Make sure the destructor of the child objects clean correctly the objects created inside it - if one of the stringlists in a child objects has owned objects, you could use the same strategy in the above item.

An (oversimplified) example:

TMyChildObject = class
private
  fMyListOfMagazineNames, 
  // This one will have TBitmap instances associated with the dogs names
  fMyListOfDogNamesAndPhotos : TStringList;
  fName: String;
public 
  //[properties, methods, whatever]

  property Name: string read fName write fName;
  constructor Create; virtual;
  destructor Destroy; override;
end;

constructor TMyChildObject.Create;
begin
  inherited Create;
  fMyListOfMagazineNames = TStringList.Create;
  fMyListOfDogNamesAndPhotos = TStringList.Create; 
  fMyListOfDogNamesAndPhotos.OwnsObjects := True;
end;

destructor TMyChildObject.Destroy;
begin
  fMyListOfMagazineNames.Free;
  fMyListOfDogNamesAndPhotos.Free;
  inherited Destroy;
end;

An example of use:

var 
  MyObjList: TStringList;
begin
  MyObjList := TStringList.Create;
  MyObjList.OwnsObjects := True;

  // Create your child objects

  // And Free them at once 
  MyObjList.Free;
end;
Fabricio Araujo
  • 3,810
  • 3
  • 28
  • 43
  • As far it appeared in the [previous question](http://stackoverflow.com/questions/7193590/tstringlist-of-objects-taking-up-tons-of-memory-in-delphi-xe), all data was to be allocated and added. It did not sound like a memory leak issue. – Arnaud Bouchez Nov 07 '11 at 16:50
  • @ArnaudBouchez: I wasn't aware of that question. So this can be an reallocation problem? – Fabricio Araujo Nov 07 '11 at 16:57
  • or an allocation problem (some not freed memory during object population). – Arnaud Bouchez Nov 08 '11 at 07:12
0

As was stated in your previous question - thanks for having selected one answer.

The main solution to your problem is not to allocate all objects in memory. If your 28 MB initial data consume more than 1 GB of memory, you have a lot of redundant information.

You shall modify your algorithm as such:

  • Use a SQL database instead to process your data without memory issue (with proper indexes) - may be SQLite3 or embedded Firebird will work very well;
  • Leave the data on the input files, the memory-map them instead of allocating a copy in memory: use getter methods for each properties, to retrieve the value on the fly;
  • Use a NoSQL database (like this) which may be faster than a SQL database for inserting rows, and consume less disk space and memory;
  • Or... consider compressing the objects content (a fast compression is a good option).
Community
  • 1
  • 1
Arnaud Bouchez
  • 42,305
  • 3
  • 71
  • 159