6

I'm experimenting with shared libraries to build a modularized program.

There are two cpp files to compile:

Shared library, compile with

g++ -fPIC -shared module.cpp -o module.so

//module.cpp
#include <iostream>

File using the shared library, compile with

g++ src/main.cpp -ldl -o binary

or

g++ -DFIX src/main.cpp -ldl -o binary

//main.cpp
#include <dlfcn.h>
#ifdef FIX
# include <iostream>
#endif

int main()
{
   void* h = dlopen("./module.so", RTLD_LAZY);
   if ( h )
   {
      dlclose(h);
   }
}

With FIX undefined, valgrind reports lot's of still reachable memory (5,373bytes), with FIX defined, no memory is leaked.

What's the problem with using iostream in shared libraries?

This problem occurs with g++-4.6, g++-4.7 and g++-4.8. g++-4.4 does not show this behaviour. I don't have other compilers to test with, sadly (and I don't want to switch to g++-4.4 because of this).

Update:

Compiling the shared library file with the additional flags -static-libstdc++ -static-libgcc reduces the number of leaked blocks, but not completely. -static-libgcc alone has no effect, -static-libstdc++ has some effect, but not as much as both.

stefan
  • 10,215
  • 4
  • 49
  • 90
  • I'm pretty sure you don't need to cast your `Module*` to a `ModuleCore*` to `delete` it, `ModuleCore` derives from `Module` and the base class has a virtual destructor. – SirGuy Oct 04 '13 at 17:15
  • @GuyGreer That's not the point of the question. – stefan Oct 04 '13 at 17:47
  • It was just a remark, I realize it doesn't relate to the question – SirGuy Oct 04 '13 at 17:48
  • @GuyGreer I know, but I also think you're wrong (I'm not sure): See this: http://stackoverflow.com/a/3261770/985296 – stefan Oct 04 '13 at 17:49
  • @GuyGreer Ah, no, I'm wrong. If I don't tamper with the pointer, i.e. something like `Module* mptr = /**/; Module m = *mptr; mptr = &m;`, then everything will be fine, even without the cast. However, the cast certainly does no harm, doesn't it? – stefan Oct 04 '13 at 17:53
  • Certainly not in this example, no – SirGuy Oct 04 '13 at 17:55
  • 2
    The biggest question is: why are you so hung up about freeing the memory on application exit - the memory that will be freed microseonds later by the operating system itself? It's wholly pointless unless you're after satisfying your curiosity only. In the latter case, well, the debugger is your friend. Trace what memory is allocated and where it's freed, and you should then see what runtime code invokes the freeing, and can even figure out why it's not freed. Debuggers, they work. – Kuba hasn't forgotten Monica Oct 06 '13 at 19:08
  • 2
    Basically, memory leaks are leaks only while the application is running. They are not leaks anymore when the application is exiting. In other words: if it's supposed to be freed during normal operation of the application, you've got a real leak. If it's freed when the application is committing to exiting, you don't worry about it. – Kuba hasn't forgotten Monica Oct 06 '13 at 19:09
  • 1
    @KubaOber That's exactly the attitude I hate. I don't care what the OS does _after_ the execution of my app. I care about my app. And if there's any little bit causing crap to happen (leaks, UB), I want to eliminate it. Noone ever should write code that leaks. It's a source of errors. – stefan Oct 06 '13 at 19:18
  • @stefan, but "still reachable" is not a leak. The memory is reachable, and theoretically still in use. If it comes from the std lib you should probably stop fretting about it. – Jonathan Wakely Oct 06 '13 at 19:28
  • @JonathanWakely This just isn't reasonable to me. I just found another "fix" right now: `int bar[1];` in main, no includes except `dlfcn.h` makes the statistic perfectly clean. I want to understand that and utimately find the correct solution such that I won't have to build workarounds. "Still reachable memory" is avoidable in this case and I want a stable solution. – stefan Oct 06 '13 at 19:36
  • 2
    @stefan: I think you're blissfully unaware that Valgrind comes with a whole bunch of "ignore this and that". What you're seeing is basically something missing from the list. When you disable those suppressions, the output will be so verbose as to be on the verge of being useless. This suppression list is full of exactly the things you're so hung up upon, and it is basically created by running reasonable programs that don't have leaks in *your* code, and adding all of the library leaks to the suppressions. – Kuba hasn't forgotten Monica Oct 06 '13 at 20:58
  • 1
    @KubaOber I know that valgrind does generate false positives, it's just another software not working perfectly (understandable, because it's incredibly complex). I'm a perfectionist w.r.t. software. If I can understand a problem, I solve it. If I can't, I ask until I understand, then solve it. This is such a problem. – stefan Oct 06 '13 at 21:35
  • @stefan: You're hung up on an arbitrary piece that's missing from the suppression list. I don't think it's a very rational thing to do. If you really want to clean it up, you need to turn off suppressions and investigate every non-freed memory area that's listed. You might spend a couple months doing that, in the worst case, for no benefit. If you want to be a perfectionist, make sure your application doesn't block if it's a GUI application: don't do file I/O in the GUI thread, understand how to use your framework's threading and inter-thread communication primitives to your advantage, etc. – Kuba hasn't forgotten Monica Oct 07 '13 at 11:21
  • @KubaOber GUI development is completely off topic here. – stefan Oct 07 '13 at 11:37
  • @stefan: It was just an example. What I meant generally was that you have to choose your battles, and fight those where winning is worth it to your customer at least. – Kuba hasn't forgotten Monica Oct 07 '13 at 13:46
  • @KubaOber Well yes, most people have different priorities. If you think it's not worth fighting here, move along, don't waste anyones time, including yours. I think I made it clear enough what my goal is here. – stefan Oct 07 '13 at 13:55
  • I can not reproduce this with g++ 4.7.3. I would just add suppression. BTW you are missing `#include ` in main.cpp – BЈовић Oct 08 '13 at 07:40
  • @BЈовић Thanks for pointing that missing include out to me (I just fixed that). I hate suppression as they still leave some junk in the statistics – stefan Oct 08 '13 at 07:53

1 Answers1

1

1. Why or how does this code snippet "fix" the issue?

I'm not sure why without digging into the libstdc++ code, but I assume that the memory allocated by the iostreams library, which is kept allocated for the duration of the whole program, is reported as a problem by valgrind when it is allocated in a shared library, but not when allocated in the main program.

2. What equivalent, independent (from the standard library) code snippet provides the same bug fix?

Firstly, I don't know why you want something "independent from the standard library" when the still reachable memory is probably being allocated by the standard library. The fix is either don't use the standard library at all, anywhere, or use it differently.

Secondly, that "fix" is undefined behaviour, because you violate the One Definition Rule by redefining std::ios_base differently to the proper definition in the std lib.

The correct way to get the same behaviour is to #include <iostream> in your main.cpp file, including <iostream> defines a static std::ios_base::Init object. Alternatively, just #include <ios> and then define the static variable (but don't redefine the std::ios_base type) but that's basically what <iostream> does, so you might as well use that.

Jonathan Wakely
  • 166,810
  • 27
  • 341
  • 521
  • it's nothing to do with glibc, it's libstdc++. My initial answer was missing the answer to the first question, I've put it back in now – Jonathan Wakely Oct 06 '13 at 16:44
  • Well if the problem doesn't involve `module_core.so` (which includes ``) you should have shown the further reduced example in the first place – Jonathan Wakely Oct 06 '13 at 16:48
  • 1
    I can't reproduce the valgrind results using G++ 4.7.3 or 4.8.1 with either your original `module_core.cpp` code or using `dlopen(NULL, RTLD_LAZY)` – Jonathan Wakely Oct 06 '13 at 16:57
  • ... so maybe you're right and it's caused by your glibc, or you're missing a required valgrind suppressions file for the glibc allcoations, in which case I have no idea why the `ios::Init` object changes anything – Jonathan Wakely Oct 06 '13 at 17:03
  • you're right about `iostream` causing the issue, I get that now. I've completely rewritten my question to address that. – stefan Oct 06 '13 at 17:14