4

I have a shared library that gets initialized by a call to the following function:

extern "C" {

int pa__init(pa_module *m) {
    m->userdata = new PAModule(m);
    return 0;
}

} // extern "C"

It gets compiled into this (the addresses are compile time offsets):

0000000000064717 <pa__init>:
   [...]
   64726:       bf 40 00 00 00          mov    $0x40,%edi

   6472b:       e8 e0 cb ff ff          callq  61310 <operator new(unsigned long)@plt>

   64730:       48 89 c3                mov    %rax,%rbx
   64733:       48 8b 45 d8             mov    -0x28(%rbp),%rax
   64737:       48 89 c6                mov    %rax,%rsi
   6473a:       48 89 df                mov    %rbx,%rdi
   6473d:       e8 2e e1 ff ff          callq  62870 <PAModule::PAModule(pa_module*)@plt>
   [...]

This is the disassembly of the PLT function at compile time offset 61310:

0000000000061310 <operator new(unsigned long)@plt>:
   61310:       ff 25 a2 c1 66 00       jmpq   *0x66c1a2(%rip)        # 6cd4b8 <operator new(unsigned long)@@Base+0x57f708>
   61316:       68 94 02 00 00          pushq  $0x294
   6131b:       e9 a0 d6 ff ff          jmpq   5e9c0 <.plt>

When I am loading the library and this method gets called, I get a segfault at compile time offset 61310:

#0  0x0000000000061316 in ?? ()
#1  0x00007fd9faae9730 in pa__init (m=0x55f1a750d850) at pa_module.cpp:24
[...]

The value in the GOT at compile time offset 6cd4b8 (relocated at runtime to e.g. 0x7fced7ffa4b8) is

0x7fced7ffa4b8: 0x00061316

The processor tries to jump to that location. However, this is still the compile time offset (pointing to invalid memory), which is the reason why the program segfaults.

Any ideas why the entries in the GOT do not get relocated when my library is loaded?

Thank you very much!

Sämy
  • 799
  • 2
  • 7
  • 17
  • `0x0000000000061316` is the `push`-immediate instruction. It should only segfault on a stack overflow, or if `rsp` is pointing somewhere bogus. **Did the caller maybe break the stack, e.g. with a very large local array or `alloca`?** Can you confirm that `RIP` was really pointing at the `push` instruction? From your backtrace, it doesn't look like you actually segfaulted during lazy dynamic linking. – Peter Cordes Nov 19 '17 at 23:36
  • i.e. check with gcc that the memory pointed to by the stack pointer is readable. Use `x $rsp` to dump starting at that address. – Peter Cordes Nov 19 '17 at 23:39
  • Actually, when I step through the code I only reach line `61310`. At that point all the registers seem to be valid. The next step already results in the segfault. Also, the jump ends up going to `0x7fced7ffa4b8`. I added the GOT at this memory location to the question. – Sämy Nov 19 '17 at 23:59
  • Ah, there's your problem. `add %al, (%rax)` is what you get when you disassemble `00 00`. The memory you jumped to isn't code, it's data of some sort. Maybe look in your /proc/PID/smaps to find out which part of your process that address (`0x7fced7ffa4b8`) is in. Now, the question is how that pointer got into the GOT entry. Can you give more details on how you built your library and linked the executable? (compiler version and options, and whether `-pie` is enabled by default...). – Peter Cordes Nov 20 '17 at 00:07
  • It seems as if that memory belongs to my library... `-PIE` is not enabled but `-PIC` is. – Sämy Nov 20 '17 at 00:29
  • It does belong to your library, but it's data not code (no `x` permission) – Jester Nov 20 '17 at 00:31
  • I see... How does that happen? :) – Sämy Nov 20 '17 at 00:32
  • 1
    Are you using `dlopen` to load it? Try the `RTLD_NOW` flag. Also, put a write watch on `0x6cd4b8` to see who is modifying it. – Jester Nov 20 '17 at 00:34
  • 1
    @Sämy: I was asking how the executable *calling* your library was linked. It's normal (and the only correct option) for your library code to be build with `-fPIC`. And BTW, `-pie` is not the same as `-fPIE`, and I don't think there is a `-PIC` option, only `-fPIC`. See https://stackoverflow.com/questions/43367427/32-bit-absolute-addresses-no-longer-allowed-in-x86-64-linux for more about `-pie` for position-independent executables (enables ASLR for the executable as well as libraries). – Peter Cordes Nov 20 '17 at 00:42
  • Also I see your code is built with C++14, but is PulseAudio also built that way? If you try to mix C++ ABI versions, terrible, terrible things will happen. – Zan Lynx Nov 20 '17 at 01:22
  • @PeterCordes Sorry, I misunderstood. The caller is PulseAudio. I usually have it installed from the repos. But I build it manually on my computer which results in the compiler flags in the question. – Sämy Nov 20 '17 at 01:48
  • @ZanLynx No, I do not think that it is build with C++14. But I think it is C only. Does this make any difference? – Sämy Nov 20 '17 at 01:48
  • @Jester PulseAudio uses `lt_dlopen` to load my library. But when should I put this write watch? Because `0x6cd4b8` is the PIC offset. Does GDB do the relocation as well? – Sämy Nov 20 '17 at 01:52
  • 2
    Oh, it's pulse loading your module. That's unfortunate. Can you `LD_PRELOAD` it? Not sure if that affects `dlopen`. Also set `LD_BIND_NOW` environment variable. – Jester Nov 20 '17 at 01:55
  • *the gdb disassemble of the GOT* That's disassembly of memory *pointed to* by the GOT entry, right? i.e. the jump target address that the indirect `jmp` will load from `0x6cd4b8` using a `0x66c1a2(%rip)` addressing mode. Except for some reason you disassembled from some earlier address instead of the actual jump target?? x86 instructions are variable length and can start at any position. You have to disassemble from the right starting address for disassembly to be "in sync" with how the CPU will decode after jumping. Use `disas` with a start address. – Peter Cordes Nov 20 '17 at 01:55
  • Or just use `layout asm` / `layout reg` in GDB, so you can see the instructions at and after RIP when you're stopped while single-stepping or on a segfault. (But I don't think correct disassembly of the jump target will solve anything, the problem is you're jumping to a non-executable page so the problem is earlier.) – Peter Cordes Nov 20 '17 at 01:56
  • @PeterCordes Ouch, I forgot that `jmp` is indirect... The value at memory location `0x6cd4b8` (relocated at runtime) is `0x00061316` hence the segfault. Now the question remains why this table is not updated with the relocated function addresses... – Sämy Nov 20 '17 at 02:21
  • @Sämy: `0x00061316` is the push instruction. This is how lazy dynamic linking works: initially the GOT entries point back into the PLT, to run the `push` and `jmp` to the dynamic linker's symbol-resolution code. That code updates the GOT entry (as well as eventually jumping to the actual function), so future calls through this PLT entry jump directly to the function call from the first `jmp`. This is why Jester is suggesting you try early binding (where all the GOT entries are resolved at dynamic link time, instead of waiting for them to be used) with `LD_BIND_NOW=1 ./a.out` or something. – Peter Cordes Nov 20 '17 at 03:18
  • So no, that doesn't explain the segfault at all. The load from `0x7fced7ffa4b8` is a data load (of a pointer), not a code fetch, so the page containing `0x7fced7ffa4b8` does *not* need execute permission, only read. – Peter Cordes Nov 20 '17 at 03:19
  • 2
    @PeterCordes But it does. Because at runtime `0x7fced7ffa4b8` still holds the non-relocated address `0x61316`. This is not (no longer) the push instruction at runtime. (`$rip=0x7f8faf77d2f0 @ 0x61310` but `*0x7fced7ffa4b8 = 0x61316` and not `0x7f8faf77d2f6`). – Sämy Nov 20 '17 at 03:25
  • 1
    Ah I see. That wasn't clear *at all* from the question, because some of it's from gdb with runtime addresses (including `0x61316` as a runtime address), and you even say "I get a segfault at line `6472b`" which is deceptive. (and not accurate, because it's actually from the PLT that `call` jumped to). Also, you show a block from `/proc//smaps` as if it was a big deal, which makes it look like you're confused. Anyway, `gdb` makes it easy to show relocated disassembly with runtime addresses (`disas /r`). I'd suggest updating the question so others aren't confused the way I way. – Peter Cordes Nov 20 '17 at 03:49
  • And BTW, yes I realize now that `0x61316` is too low for a normal code address, but only by 1 digit. In a position-dependent executable for x86-64, `_start` is typically at `4000c0`, i.e. just above 4MiB. So I didn't think (until you pointed it out) that `0x61316` was too low to be a real runtime address, and was just relative to the start of the library. – Peter Cordes Nov 20 '17 at 03:56
  • @PeterCordes You are right, I updated the question. In the meantime I realized that some symbols were undefined in my library because I mixed up code that is compiled with Clang++ and g++. I think that was the reason why the relocation in the GOT was not performed successfully. – Sämy Nov 22 '17 at 13:59
  • Thank you @PeterCordes for your support! – Sämy Nov 22 '17 at 14:08
  • 2
    @Jester Thank you as well. `LD_BIND_NOW=1` helped me to realize that there were undefined symbols in my library. – Sämy Nov 22 '17 at 14:09

1 Answers1

3

My library turned out to have undefined symbols.

This prevented the dynamic linking loader to successfully update the GOT and the compile time offsets were kept in memory. At runtime these addresses are invalid and resulted in the segfault.

Sämy
  • 799
  • 2
  • 7
  • 17