2

When I'm trying to use the DYLD_INSERT_LIBRARIES environmental variable to insert a .dylib file to a running process on macOS Mojave, I'm encountering a segmentation fault.

System version: macOS 14.4.4

Compiler version:

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

debug_malloc.c (compiles to the dylib file):

#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdio.h>
#include <stdlib.h>


#define DYLD_INTERPOSE(_replacment,_replacee) \
__attribute__((used)) static struct{ const void* replacment; const void* replacee; } _interpose_##_replacee \
__attribute__ ((section ("__DATA,__interpose"))) = { (const void*)(unsigned long)&_replacment, (const void*)(unsigned long)&_replacee };


void* pMalloc(size_t size) //would be nice if I didn't have to rename my function..
{
   printf("Allocated: %zu\n", size);
   return malloc(size);
}

DYLD_INTERPOSE(pMalloc, malloc);

The running process is a test program written in C, which does nothing except calling malloc once: test.c (compiles to test)

#include <memory.h>
#include <stdlib.h>
#include <stdio.h>
int main(void)
{
printf("before malloc\n");
    void *a=  malloc(900);
  return 0;
}

Compile & execute commands

gcc -odbg.dylib -dynamiclib ./debug_malloc.c
gcc -otest ./test.c
DYLD_INSERT_LIBRARIES=./dbg.dylib ./test

Running the last command yields 

Segmentation fault: 11

Whereas just running test without dylib preload works fine.

BearAqua
  • 518
  • 4
  • 15
  • 1
    What happens if you remove `printf` from `pMalloc`? – Eric Postpischil May 02 '19 at 18:10
  • 1
    @EricPostpischil Ah! Thanks for the comment. It worked. Am I right in saying that since `DYLD_INSERT_LIBRARIES` works like `LD_PRELOAD` and inserts the dynamic library before stdlib, calling a stdlib function results in a segfault? (So loading order matters?) – BearAqua May 02 '19 at 18:14
  • 1
    My suspicion was that `printf` uses `malloc`, so calling `printf` inside `pMalloc` results in a call to `malloc`, which gets interposed and sent to `pMalloc`, resulting in a new call to `printf`,… – Eric Postpischil May 02 '19 at 18:20
  • @EricPostpischil Oof. That's probably true. On the other hand, is there a way to print from `pMalloc` without causing infinite recursion? – BearAqua May 02 '19 at 18:22
  • 1
    `fputs` might be safer. Also, `pMalloc` could start in an inactive state in which it does nothing. In `main`, call `printf` to get it started (it may use `malloc` to set things up the first time it is called and not, usually, later on), then change the state so that `pMalloc` does print (e.g., have a routine `ChangeState` that sets a local static flag to on). But these are kludges. Search for “function interposition/interpose/interposing” to find other ideas. – Eric Postpischil May 02 '19 at 18:26
  • Another ideas is that `pMalloc` could keep a local static counter to tell whether it is in a nested call. If so, it would do nothing but pass the request on to `malloc`. E.g., `static int counter = 0; if (0 == counter++) { do stuff } else { just do plain malloc } counter--;`. That is not safe for threading etc., but it might serve your purposes. – Eric Postpischil May 02 '19 at 18:26
  • @EricPostpischil Thanks! Mind if you write these into an answer so I can accept it? – BearAqua May 02 '19 at 18:27
  • I am just spitballing, not checking enough to provide a definite answer, so I do not want to commit to an answer. Give it some time, others may answer. – Eric Postpischil May 02 '19 at 18:28
  • [This question](https://stackoverflow.com/questions/14999090/function-interposition-only-working-for-malloc-not-free) has a comment with links to a few others. – Eric Postpischil May 02 '19 at 18:30

0 Answers0