7

When working with windows dll's, it is established that we should limit the memory allocation/deallocation within the dll boundary since the dll could be using its own heap. So we have export allocators and free functions to from dll.

IsomeInterface* getObject();
void freeObject(IsomeInterface *object);

This way the object creation and deletion will reside within the dll.

Does this problem also exist on shared libraries on linux? When dealing with shared libs (.so) do we also need to take care of keeping allocation/deallocation within the shared lib. I did some quick tryout below and it works on linux. The same example would not work with windows dll (it will work with windows dll if both exe and dll compiled with /MD to make use of the same heap).

std::vector<int> vec;
vec.push_back(42);
PassAVector(vec);

where PassAVector resides in a shared lib

void PassAVector(std::vector<int> &vec)
{
     vec.push_back(1);  // These would cause reallocation
     vec.push_back(2);
     vec.push_back(3);
     vec.push_back(4);
     vec.push_back(5);
}

Does this mean shared libs on unix share the heap with the executables (equivalent to the /MD switch on windows)?

Is it possible to compile (some compiler flags) the shared lib (.so) or the executable on linux in such a way that they start using different heaps (/MT switch on windows) and this problem surfaces?

EDIT: Found this which seem to suggest passing STL across boundaries is okay as long long as the compiler is gcc.

user3819404
  • 611
  • 6
  • 18
  • `PassAVector` suffers from a different issue, that's unrelated to the OS: C++ does not have a standard ABI. Passing C++ objects across module boundaries is never supported. – IInspectable Oct 28 '20 at 15:17
  • @IInspectable Not even if both the modules are compiled with the same compiler version and flags and use the same heap? – user3819404 Oct 28 '20 at 15:23
  • If you compile both sides of the API with the exact same compiler using the exact same compiler options it's safe to pass C++ objects across (assuming they also agree on the allocator when crossing module boundaries). Though since you are compiling a library you do not control the caller, so all bets are off. – IInspectable Oct 28 '20 at 16:13
  • @IInspectable Agree. Different compiler (or versions) means two modules seeing different implemetation of say `std::vector` and hence all bets are off. Here specifically I wanted to know about the heap. On windows the modules can have different heaps and even having same compiler/config won't help if allocation crosses boundaries. What is case wrt heap with gcc? – user3819404 Oct 28 '20 at 16:19
  • I'm not familiar with implementation details of shared objects in Linux, so I cannot comment on that. A well-established solution to this issue on Windows is COM (or its successor, the Windows Runtime). This allows you to freely exchange objects across module boundaries (or processes, or computers). – IInspectable Oct 28 '20 at 17:46
  • With Windows you do not need this practice anymore because you compile to the Universal C-Runtime since VC++ 2017 (where it did really work). https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/. Then you can even mix debug/release binaries while at least with regards to memory allocation everything will work. – Alois Kraus Nov 05 '20 at 19:29

2 Answers2

4

As long as you stick with Glibc or other "normal" allocators (jemalloc, tcmalloc, etc.) the heap state will be shared by all libraries and thus you'll be able to free memory allocated somewhere with malloc anywhere you want.

In theory it may be possible to circumvent this. For example some library may be linked with custom implementation of malloc/free (via symbol scripts trickery of -Bsymbolic) which has it's own private heap and thus will not interact well with other parts of your program. But I've never seen anything like this in real life.

STL containers are based on malloc/free so it is possible to pass/modify them across library boundaries as well. Of course different libraries may be compiled with different compilers and different incompatible versions of STL (e.g. libstdc++, libcxx, etc.) but their C++ container types would be different and compiler simply would not allow you to pass them across incompatible modules.

yugr
  • 19,769
  • 3
  • 51
  • 96
1

In C++ the normal way to change the heap is to overload the global operator new/delete.

In Windows that overload is restricted to the specific dll and thus you get the problems.

In Linux the overloaded global operator is normally truly global - so the first one wins, but if you really want specific heaps in them you can achieve that (and then you need the exported allocate and free-functions):

C++ custom global new/delete overriding system libraries

Hans Olsson
  • 11,123
  • 15
  • 38