15

I use boost::shared_ptr in my application in C++. The memory problem is really serious, and the application takes large amount of memory.

However, because I put every newed object into a shared_ptr, when the application exits, no memory leaking can be detected.

There must be something like std::vector<shared_ptr<> > pool holding the resource. How can I know who holds the shared_ptr, when debugging?

It is hard to review code line by line. Too much code...

Alexis Wilke
  • 19,179
  • 10
  • 84
  • 156
user25749
  • 4,825
  • 14
  • 61
  • 83

8 Answers8

23

You can't know, by only looking at a shared_ptr, where the "sibling pointers" are. You can test if one is unique() or get the use_count(), among other methods.

13

The popular widespread use of shared_ptr will almost inevitably cause unwanted and unseen memory occupation.

Cyclic references are a well known cause and some of them can be indirect and difficult to spot especially in complex code that is worked on by more that one programmer; a programmer may decide than one object needs a reference to another as a quick fix and doesn't have time to examine all the code to see if he is closing a cycle. This hazard is hugely underestimated.

Less well understood is the problem of unreleased references. If an object is shared out to many shared_ptrs then it will not be destroyed until every one of them is zeroed or goes out of scope. It is very easy to overlook one of these references and end up with objects lurking unseen in memory that you thought you had finished with.

Although strictly speaking these are not memory leaks (it will all be released before the program exits) they are just as harmful and harder to detect.

These problems are the consequences of expedient false declarations:

  1. Declaring what you really want to be single ownership as shared_ptr. scoped_ptr would be correct but then any other reference to that object will have to be a raw pointer, which could be left dangling.
  2. Declaring what you really want to be a passive observing reference as shared_ptr. weak_ptr would be correct but then you have the hassle of converting it to share_ptr every time you want to use it.

I suspect that your project is a fine example of the kind of trouble that this practice can get you into.

If you have a memory intensive application you really need single ownership so that your design can explicitly control object lifetimes.

With single ownership opObject=NULL; will definitely delete the object and it will do it now.

With shared ownership spObject=NULL; ........who knows?......

Alexis Wilke
  • 19,179
  • 10
  • 84
  • 156
John Morrison
  • 487
  • 4
  • 3
  • "_a programmer may decide than one object needs a reference to another as a quick fix and doesn't have time to examine all the code to see if he is closing a cycle._" He doesn't have to "read code". He should read the **design documents**. – curiousguy Oct 13 '11 at 14:38
  • 8
    in the real world a lot of projects have no concept of design documents :) – paulm Oct 24 '14 at 06:41
  • 6
    @curiousguy I prefer code to design documents as documents can lie, and code compiles. – M2tM Oct 08 '15 at 00:32
  • @M2tM Comments can lie. Variable names can lie (esp. if you try to put type annotation in variable names). After changes in design, function names very often lie or are at least misleading or useless. Code can contain traces of past versions which as useful as the legs of a snake. Code that evolved a lot is often misleading. – curiousguy Oct 08 '15 at 03:00
  • 6
    Code certainly can be misleading! It's not an easy task to make a well written piece of software, and it's even harder (or should I say, requires continuous effort) to change an existing system while maintaining quality. But comments and documentation don't compile, and are often the first things to get out of sync with the functionality of the thing they attempt to describe. Certainly variable names and function names can be poorly named, but it seems less common in the code bases I've encountered in my career. Typically the design document is written before code and never updated. – M2tM Oct 08 '15 at 18:27
9

One solution to dangling or circular smart pointer references we've done is customize the smart pointer class to add a debug-only bookkeeping function. Whenever a smartpointer adds a reference to an object, it takes a stack trace and puts it in a map whose each entry keeps track of

  1. The address of the object being allocated (what the pointer points to)
  2. The addresses of each smartpointer object holding a reference to the object
  3. The corresponding stacktraces of when each smartpointer was constructed

When a smartpointer goes out of scope, its entry in the map gets deleted. When the last smartpointer to an object gets destroyed, the pointee object gets its entry in the map removed.

Then we have a "track leaks" command with two functions: '[re]start leak tracking' (which clears the whole map and enabled tracking if its not already), and 'print open references', which shows all outstanding smartpointer references created since the 'start leak tracking' command was issued. Since you can see the stack traces of where those smart pointers came into being, you can easily know exactly who's keeping from your object being freed. It slows things down when its on, so we don't leave it on all the time.

It's a fair amount of work to implement, but definitely worth it if you've got a codebase where this happens a lot.

vercellop
  • 553
  • 5
  • 18
3

You may be experiencing a shared pointer memory leak via cycles. What happens is your shared objects may hold references to other shared objects which eventually lead back to the original. When this happens the cycle keeps all reference counts at 1 even though no one else can access the objects. The solution is weak pointers.

deft_code
  • 57,255
  • 29
  • 141
  • 224
  • Thank you very much! I really use weak_ptr as a resource watcher. So I know large amount of shared_ptr<> is existing in memory. I am sure there is no cycles, some module is ill designed, I am trying to find out it. – user25749 Jun 30 '09 at 06:40
  • 3
    "_The solution is weak pointers._" No it isn't. The solution is to review the design. – curiousguy Oct 10 '11 at 14:20
  • @curiousguy: Yes yes, to be pedantic the _real_ solution to cyclic ownership is to avoid it. However, there are legitimate problems that are better solved with cyclic pointers. In those cases: A solution to a memory leak caused by a truly needed cyclic reference is to use `std::weak_ptr`. – deft_code Oct 11 '11 at 20:08
  • @deft_code You had a cycle of strong references implemented with `shared_ptr`. If you replace one `shared_ptr` with a `weak_ptr`, you no longer have a cycle of strong references; IOW: **you don't have a cycle of owning smart pointers** any more. You have "solved" the cyclic ownership problem by suppressing cyclic ownership. If the "solution" is acceptable, it means that **you actually never needed the semantic of cyclic ownership**. `weak_ptr` is _not_ magic. – curiousguy Oct 13 '11 at 14:36
3

Try refactoring some of your code so that ownership is more explicitly expressed by the use of weak pointers instead of shared pointers in some places.

When looking at your class hierarchy it's possible to determine which class really should hold a shared pointer and which merely needs only the weak one, so you can avoid cycles if there are any and if the "real" owner object is destructed, "non-owner" objects should have already been gone. If it turns out that some objects lose pointers too early, you have to look into object destruction sequence in your app and fix it.

macbirdie
  • 16,086
  • 6
  • 47
  • 54
  • What you are saying is that **weak references can be used for debugging**: if the weak reference is dead we need to use it, it means the program has a bug. This is the first time I see this idea clearly expressed on OS - some other posters uses of `weak_ptr` seem to imply that it is a debugging tool, but they don't say that clearly. Remark: that is less efficient than using a regular C++ pointer: maybe we need a `checked_ptr` that can be defined as `weak_ptr` or as a "regular pointer" smart pointer. – curiousguy Oct 10 '11 at 14:38
1

You're obviously holding onto references to your objects within your application. This means that you are, on purpose, keeping things in memory. That means, you don't have a memory leak. A memory leak is when memory is allocated, and then you do not keep a reference to its address.

Basically, you need to look at your design and figure out why you are keeping so many objects and data in memory, and how can you minimize it.

The one possibility that you have a pseudo-memory leak is that you are creating more objects than you think you are. Try putting breakpoints on all statements containing a 'new'. See if your application is constructing more objects than you thought it should, and then read through that code.

The problem is really not so much a memory-leak as it is an issue of your application's design.

Kieveli
  • 10,944
  • 6
  • 56
  • 81
  • 1
    Many thanks! There are about 200 thousands lines. So It is hard to check every new... is there any compiler Macro to activate boost's ref check ability(if such ability is exist). The memory is causing by programing logic error about pools, I am sure, but I just can not find it. – user25749 Jun 30 '09 at 04:09
  • 3
    You can still have memory leaks with shared_ptrs. Create a cyclic reference, and it'll never get deleted, even if the rest of the app no longer references it. Instant memory leak! – jalf Jun 30 '09 at 11:32
  • 5
    An reference to an object being incorrectly retained is still a resource leak. This is why GC programs can still have leaks, usually due to the Observer pattern - the observer is on a list instead the observable and never gets taken off it. Ultimately, a `remove` is needed for every `add`, just as a `delete` is needed for every `new`. Exactly the same programming error, causing exactly the same problem. A "resource" is a really just a pair of functions that have to be called an equal number of times with corresponding arguments, and a "resource leak" is what happens when you fail to do that. – Daniel Earwicker Aug 07 '09 at 08:24
1

I was going to suggest using UMDH if you are on windows. It is a very powerful tool. Use it find allocations per transaction/time-period that you expect to be freed then find who is holding them.

There is more information on this SO answer Find memory leaks caused by smart pointers

Community
  • 1
  • 1
iain
  • 10,798
  • 3
  • 37
  • 41
0

It is not possible to tell what objects own shared_ptr from within the program. If you are on Linux, one sure way to debug the memory leaks is the Valgrind tool -- while it won't directly answer your question, it will tell where the memory was allocated, which is usually sufficient for fixing the problem. I imagine Windows has comparable tools, but I am do not know which one is best.

Vladimir Prus
  • 4,600
  • 22
  • 31