Replacing malloc via shared library

Question

I'm working on a cmake based project in Unix environment. I've created a shared library myMalloc.so where I've defined my own malloc/free/realloc etc.:

//   myMalloc.cpp

extern "C" __typeof (malloc) __libc_malloc;
extern "C" {

void* malloc( size_t _size )
{
    /*some code that allows me to determine if this version of malloc was called*/
    return __libc_malloc( _size );
}
}

Some other shared library proxy.so is linked with myMalloc.so. Finally proxy.so is linked with the executable binary.

The issue is that default glibc malloc is used in the executable instead of my own malloc.

I've tried adding below flags into multiple different CMakeLists.txt files

set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free")

but it didn't help.

I've checked readelf -Ws myMalloc.so and can see my version of malloc marked as GLOBAL, shown before any other malloc

58: 0000000000017110    79 FUNC    GLOBAL DEFAULT   11 malloc
255: 0000000000017110    79 FUNC    GLOBAL DEFAULT   11 malloc
304: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __libc_malloc

Then I've checked readelf -Ws proxy.so and sadly, glibc malloc is visible earlier than my custom malloc (I'm guessing that it means glibc version will be preferred)

72: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND malloc@GLIBC_2.2.5 (10)
450: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND malloc

readelf -Ws on the executable doesn't list any version of malloc

Compiling myMalloc as static lib doesn't help, but compiling both myMalloc and proxy as static libs seems to help (malloc is also then visible on symbol table of the executable).

Sadly, this is not an option and these libs need to remain shared and I cannot use LD_PRELOAD trick

What could be wrong ? I'm guessing that some other library which uses glibc malloc is linked earlier, but I have no idea how to approach this. I hoped that flags in cmake mentioned earlier would help, but apparently they don't.

I've spent a few days digging through the internet over this as my makefiles/linker knowledge is somewhat limited and couldn't find much more info, I'll appreciate any suggestions.

Edit: I don't see how the marked duplicate question answers my question.. I have a specific issue related with shared libraries while linking isn't even mentioned in the duplicate.

Yes, none of the duplicates actually addresses the problem in the question. Reopened. — Tsyvarev, Jun 14 '19 at 20:29
If you are on linux, you may be interested in the `--wrap` option of `ld`. The manual describes an example precisely about `malloc`. — Marc Glisse, Jun 14 '19 at 20:52
https://stackoverflow.com/questions/14168388/how-to-replace-default-malloc-by-code, https://stackoverflow.com/questions/12172988/in-c-is-it-possible-to-change-exported-function-name-to-different-one/12173140#12173140, https://stackoverflow.com/questions/19023018/overriding-c-library-functions-calling-original. Do none of these methods work for you? — fdk1342, Jun 16 '19 at 17:13
https://www.gnu.org/software/libc/manual/html_node/Replacing-malloc.html just LD_PRELOAD it. — KamilCuk, Jun 30 '19 at 08:00

Replacing malloc via shared library

0 Answers0