Overriding memory allocator in MSVC++

Question

While Microsoft standard runtime provides debugging version of allocation functions, it does not really work, because you are not supposed to use naked new in C++ code, so the instrumentation points to standard library, or nowhere because standard library can't be instrumented anyway.

Now I have code that can produce (and record) backtraces of allocations and I also used DUMA. However attempts at replacing the allocation functions broke when we used streams, because streambuf calls to some debug variant and does so inconsistently between new and delete.

So does anybody have experience with replacing allocator, by overriding the functions, not ugly preprocessor tricks, in Microsoft standard runtime? I suspect it involves avoiding the debug allocator, but I want to keep the _DEBUG define for obvious reasons (a lot more debug code depends on it).

Note: we are currently stuck with Visual C++ 9.0 (Visual Studio 2008).

Edit: Avoiding the debug allocator is unlikely to be an option, because the C++ standard library needs to have consistent definitions of new and delete between the functions and instantiations compiled to the library and the instantiations generated in user code since allocation might be done by one and release by the other. That by the way means that defining static inline variants in force-included header is unlikely to cut it.

Edit2: It's not possible with dynamic linking, because Windows bind symbols from specific DLLs, so there is no way to override them at link-time. But we don't need dynamic linking and don't use it because the primary target is WinCE and static linking is default there.

I haven't done this in quite a few years, but it seems to me that the problem is that the runtime library's DLL contains the `operator new` and `operator delete` code; they're hard-linked within the DLL, so you can't replace them. The trick is to link to the static runtime library, and provide your own `operator new` and `operator delete`, so that the ones in the static library aren't needed. But that only works for a monolithic application; it will get you into trouble if you build your own DLL's. Welcome to DLL hell. — Pete Becker, Oct 10 '12 at 14:24
@PeteBecker: Our project is targeting mobile systems (WinCE and some other operating systems) and static linking is default on WinCE, so we always linked statically. Overriding (with `/FORCE` compiler option) mostly worked, but I remember having problem with `streambuf` doing something strange. — Jan Hudec, Oct 11 '12 at 06:47
WinCE is well named. When I worked for Dinkumware, one of the jobs I did was porting the standard library to WinCE. This was ten years ago, so probably things have changed. At that time, they had a new linker that did a Unix-like link: unlike the DOS linker, it didn't save the names that were defined in a library; it resolved names that it could, then went on to the next library, it did as many passes through the library list as it needed to resolve names. I spent weeks running verbose links and staring at the results to figure out why it was mixing up libraries. Lots of support calls, too. — Pete Becker, Oct 11 '12 at 10:41
Whoops, because of character limits I left out the most important part: the linker would sometimes scramble the order of the libraries on later passes, and sometimes create incorrect path names. I assume those problems have been fixed by now. — Pete Becker, Oct 11 '12 at 11:05

rustyx · Answer 1 · 2016-12-09T10:52:42.153

Here's how we do it (with jemalloc, but any other allocator is possible):

Compile the custom memory allocator separately as a C static library.
Link your C++ application with the custom allocator library.
Override operators new and delete in your C++ application to invoke the custom allocator.

Notes:

The custom allocator must to be written in C, not C++.
It is not possible to ensure sufficiently early initialization of the allocator unless it resides in a separate library.
Overriding malloc and free is also possible but a lot harder in MSVC, because those are not "weakly"-linked, and because there are a lot of custom variants in MSVC (for example using the /FORCE:MULTIPLE linker flag).

Sample code:

void* operator new(size_t size)
{
  void* ptr = my_malloc(size);
  if (ptr)
    return ptr;
  else
    throw std::bad_alloc();
}

void* operator new[](size_t size)
{
  void* ptr = my_malloc(size);
  if (ptr)
    return ptr;
  else
    throw std::bad_alloc();
}

void* operator new(size_t size, const std::nothrow_t&) throw()
{
  return my_malloc(size);
}

void* operator new[](size_t size, const std::nothrow_t&) throw()
{
  return my_malloc(size);
}

void operator delete(void* pointer) throw()
{
  my_free(pointer);
}

void operator delete[](void* pointer) throw()
{
  my_free(pointer);
}

void operator delete(void* pointer, const std::nothrow_t&) throw()
{
  my_free(pointer);
}

void operator delete[](void* pointer, const std::nothrow_t&) throw()
{
  my_free(pointer);
}

Where do you override the `operator new` and `operator delete`? How do you make sure they are available to all instances of the `std::allocator`? Keep in mind, that the application's own code, following good C++ practice, does **not contain any uses of `new`** except via standard library or Boost interfaces. And how do you deal with that pesky inconsistency in `std::basic_streambuf` (it's been 6 years, so my memory is somewhat rusty as to what exactly it was doing, but it was some debug function in one direction coupled with the standard one in the other)? — Jan Hudec, Dec 09 '16 at 12:47
wow, this approach works great! with it i managed to replace C++'s default memory allocator with jemalloc in my application. just put the sample codes in a header file, make sure the header get included in every cpp that demands custom memory allocator, and link it with the custom memory allocator lib, then it's done.although it would be nice if we could get the custom memory allocator enabled in link time, just link tcmalloc, however, i would say the approach presented here is almost the best we could get for now, provided you have access to the source code of course. — uwydoc, Feb 06 '18 at 07:25

Overriding memory allocator in MSVC++

1 Answers1

Linked