How shared library finds GOT section?

Question

While I was reading http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/#id1 question came:

How does PIC shared library after being loaded somewhere in virtual address space of the process knows how to reference external variables?

Here is code of shared library in question:

#include <stdio.h>

extern long var;

void
shara_func(void)
{
        printf("%ld\n", var);
}

Produce object code, then shared object(library):

gcc -fPIC -c lib1.c                    # produce PIC lib1.o
gcc -fPIC -shared lib1.o -o liblib1.so # produce PIC shared library

Disassemble shara_func in shared library:

objdump -d liblib1.so
 ...
 00000000000006d0 <shara_func>:
 6d0:   55                      push   %rbp
 6d1:   48 89 e5                mov    %rsp,%rbp
 6d4:   48 8b 05 fd 08 20 00    mov    0x2008fd(%rip),%rax        # 200fd8 <_DYNAMIC+0x1c8>
 6db:   48 8b 00                mov    (%rax),%rax
 6de:   48 89 c6                mov    %rax,%rsi
 6e1:   48 8d 3d 19 00 00 00    lea    0x19(%rip),%rdi        # 701 <_fini+0x9>
 6e8:   b8 00 00 00 00          mov    $0x0,%eax
 6ed:   e8 be fe ff ff          callq  5b0 <printf@plt>
 6f2:   90                      nop
 6f3:   5d                      pop    %rbp
 6f4:   c3                      retq   
...

I see that instruction at 0x6d4 address moves some address that is relative to PC to rax, I suppose that is the entry in GOT, GOT referenced relatively from PC to get address of external variable var at runtime(it is resolved at runtime depending where var was loaded). Then after executing instruction at 0x6db we get external variable's actual content placed in rax, then move value from rax to rsi - second function parameter passed in register.

I was thinking that there is only one GOT in process memory, however, see that library references GOT? How shared library knows offset to process's GOT when it(PIC library) does not know where in process memory it would be loaded? Or does each shared library has its own GOT that is loaded with her? I would be very glad if you clarify my confusion.

Bulat, did you check `objdump -r` and `objdump -R` searching for RELOC (relocations) in your ELF - http://man7.org/linux/man-pages/man1/objdump.1.html? — osgx, Oct 02 '16 at 06:54
@osgx, yes, that is 0x200fd8 R_X86_64_GLOB_DAT var + 0 relocation. PC-relative: 0x2008fd + 0x6db = 0x200fd8. Do not understand how linker calculated 0x2008fd offset, how it knows where process's GOT will be located in runtime. — Bulat M., Oct 02 '16 at 07:07
Bulat, the truth is: there are two linkers: one is `ld` command and other is runtime linker, RTLD, `ld-linux.so.2` program in linux (it is registered as interpreter of dynamic ELFs - http://stackoverflow.com/questions/5130654). So, R_X86_64_GLOB_DAT created by ld linker will be resolved by runtime linker RTLD: http://osxr.org:8080/glibc/source/sysdeps/x86_64/dl-machine.h#0304. It is the one who opens lib file; mmaps it to right place, create the real GOT and resolves relocations to point to real GOT. — osgx, Oct 02 '16 at 07:18
@osgs, each shared library is mapped by runtime linker(RTLD) to process's virtual address space with shared library's accompanying GOT and PLT(if needed) in its "data section", so that process could have multiple GOT for multiply shared libraries? — Bulat M., Oct 02 '16 at 07:23
Bulat, If I understand http://eli.thegreenplace.net/2011/11/03/position-independent-code-pic-in-shared-libraries/ & https://www.technovelty.org/linux/plt-and-got-the-key-to-code-sharing-and-dynamic-libraries.html correctly, yes, if the GOT is loaded like .data of every library, there are several GOTs. And the linker will put real absolute address of finally selected variable (it can resolve to other lib or to the executable) into the right entry in GOT (is there any additional RELOCs points inside `.got` section of the lib? or is it _GLOB_DAT?). `0x6db` will use this pointer to get the var. — osgx, Oct 02 '16 at 07:42
@osgx, yes, any additional extern variable references result in _GLOB_DAT relocations. — Bulat M., Oct 02 '16 at 12:39

score 10 · Accepted Answer · edited May 23 '17 at 10:34

I was thinking that there is only one GOT in process memory, however, see that library references GOT?

We clearly see .got section as part of the library. With readelf we can find what are the sections of the library and how they are loaded:

readelf -e liblib1.so
...
Section Headers:
  [21] .got              PROGBITS         0000000000200fd0  00000fd0
       0000000000000030  0000000000000008  WA       0     0     8
...

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x000000000000078c 0x000000000000078c  R E    200000
  LOAD           0x0000000000000df8 0x0000000000200df8 0x0000000000200df8
                 0x0000000000000230 0x0000000000000238  RW     200000
...
 Section to Segment mapping:
  Segment Sections...
   00     ... .init .plt .plt.got .text .fini .rodata .eh_frame_hdr .eh_frame 
   01     .init_array .fini_array .jcr .dynamic .got .got.plt .data .bss 
   02     .dynamic

So, there is section .got, but runtime linker ld-linux.so.2 (registered as interpreter for dynamic ELFs) does not load sections; it loads segments as described by Program header with LOAD type. .got is part of segment 01 LOAD with RW flags. Other library will have own GOT (think about compiling liblib2.so from the similar source, it will not know anything about liblib1.so and will have own GOT); so it is "Global" only for the library; but not to the whole program image in memory after loading.

How shared library knows offset to process's GOT when it(PIC library) does not know where in process memory it would be loaded?

It is done by static linker when it takes several ELF objects and combine them all into one library. Linker will generate .got section and put it to some place with known offset from the library code (pc-relative, rip-relative). It writes instructions to program header, so the relative address is known and it is the only needed address to access own GOT.

When objdump is used with -r / -R flags, it will print information about relocations (static / dynamic) recorded in the ELF file or library; it can be combined with -d flag. lib1.o object had relocation here; no known offset to GOT, mov has all zero:

$ objdump -dr lib1.o 
lib1.o:     file format elf64-x86-64

Disassembly of section .text:

0000000000000000 <shara_func>:
   0:   55                      push   %rbp
   1:   48 89 e5                mov    %rsp,%rbp
   4:   48 8b 05 00 00 00 00    mov    0x0(%rip),%rax        # b <shara_func+0xb>
            7: R_X86_64_REX_GOTPCRELX   var-0x4
   b:   48 8b 00                mov    (%rax),%rax
   e:   48 89 c6                mov    %rax,%rsi

In library file this was converted to relative address by gcc -shared (it calls ld variant collect2 inside):

$ objdump -d liblib1.so 

liblib1.so:     file format elf64-x86-64

00000000000006d0 <shara_func>:
 6d0:   55                      push   %rbp
 6d1:   48 89 e5                mov    %rsp,%rbp
 6d4:   48 8b 05 fd 08 20 00    mov    0x2008fd(%rip),%rax        # 200fd8 <_DYNAMIC+0x1c8>

And finally, there is dynamic relocation into GOT to put here actual address of var (done by rtld - ld-linux.so.2):

$ objdump -R liblib1.so 

liblib1.so:     file format elf64-x86-64

DYNAMIC RELOCATION RECORDS
OFFSET           TYPE              VALUE 
...
0000000000200fd8 R_X86_64_GLOB_DAT  var

Let's use your lib, adding executable with definition, compiling it and running with rtld debugging enabled:

$ cat main.c 
long var;
int main(){
    shara_func();
    return 0;
}
$ gcc main.c -llib1 -L. -o main -Wl,-rpath=`pwd`
$ LD_DEBUG=all ./main 2>&1 |less
...
   311:     symbol=var;  lookup in file=./main [0]
   311:     binding file /test3/liblib1.so [0] to ./main [0]: normal symbol `var'

So, linker was able to bind relocation for var to the "main" ELF file where it is defined:

$ gdb -q ./main 
Reading symbols from ./main...(no debugging symbols found)...done.
(gdb) b main
Breakpoint 1 at 0x4006da
(gdb) r
Starting program: /test3/main 

Breakpoint 1, 0x00000000004006da in main ()
(gdb) disassemble shara_func
Dump of assembler code for function shara_func:
   0x00007ffff7bd56d0 <+0>: push   %rbp
   0x00007ffff7bd56d1 <+1>: mov    %rsp,%rbp
   0x00007ffff7bd56d4 <+4>: mov    0x2008fd(%rip),%rax        # 0x7ffff7dd5fd8
   0x00007ffff7bd56db <+11>:    mov    (%rax),%rax
   0x00007ffff7bd56de <+14>:    mov    %rax,%rsi

No changes in mov in your func. rax after func+4 is 0x601040, it is third mapping of ./main according to /proc/$pid/maps:

00601000-00602000 rw-p 00001000 08:07 6691394                            /test3/main

And it was loaded from main after this program header (readelf -e ./main)

LOAD           0x0000000000000df0 0x0000000000600df0 0x0000000000600df0
               0x0000000000000248 0x0000000000000258  RW     200000

It is part of .bss section:

 [26] .bss              NOBITS           0000000000601038  00001038
      0000000000000010  0000000000000000  WA       0     0     8

After stepping to func+11, we can check value in GOT:

(gdb) b shara_func
(gdb) r
(gdb) si
0x00007ffff7bd56db in shara_func () from /test3/liblib1.so
1: x/i $pc
=> 0x7ffff7bd56db <shara_func+11>:  mov    (%rax),%rax
(gdb) p $rip+0x2008fd
$6 = (void (*)()) 0x7ffff7dd5fd8
(gdb) x/2x 0x7ffff7dd5fd8
0x7ffff7dd5fd8: 0x00601040  0x00000000

Who did write correct value to this GOT entry?

(gdb) watch *0x7ffff7dd5fd8
Hardware watchpoint 2: *0x7ffff7dd5fd8
(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /test3/main 

Hardware watchpoint 2: *0x7ffff7dd5fd8

Old value = <unreadable>
New value = 6295616
0x00007ffff7de36bf in elf_machine_rela (..) at ../sysdeps/x86_64/dl-machine.h:435
(gdb) bt
#0  0x00007ffff7de36bf in elf_machine_rela (...) at ../sysdeps/x86_64/dl-machine.h:435
#1  elf_dynamic_do_Rela (...) at do-rel.h:137
#2  _dl_relocate_object (...) at dl-reloc.c:258
#3  0x00007ffff7ddaf5b in dl_main (...)        at rtld.c:2072
#4  0x00007ffff7df0462 in _dl_sysdep_start (start_argptr=start_argptr@entry=0x7fffffffde20, 
    dl_main=dl_main@entry=0x7ffff7dd89a0 <dl_main>) at ../elf/dl-sysdep.c:249
#5  0x00007ffff7ddbe7a in _dl_start_final (arg=0x7fffffffde20) at rtld.c:307
#6  _dl_start (arg=0x7fffffffde20) at rtld.c:413
#7  0x00007ffff7dd7cc8 in _start () from /lib64/ld-linux-x86-64.so.2

(gdb) x/2x 0x7ffff7dd5fd8
0x7ffff7dd5fd8: 0x00601040  0x00000000

Runtime linker of glibc did (rtld.c), just before calling main - here is the source (bit different version) - http://code.metager.de/source/xref/gnu/glibc/sysdeps/x86_64/dl-machine.h

329 case R_X86_64_GLOB_DAT:
330 case R_X86_64_JUMP_SLOT:
331   *reloc_addr = value + reloc->r_addend;
332   break;

With reverse stepping we can get history of code and old value = 0:

(gdb) b _dl_relocate_object 
(gdb) r
(gdb) dis 3
(gdb) target record-full
(gdb) c
(gdb) disp/i $pc
(gdb) rsi
(gdb) rsi
(gdb) rsi
(gdb) x/2x 0x7ffff7dd5fd8
0x7ffff7dd5fd8: 0x00000000  0x00000000


=> 0x7ffff7de36b8 <_dl_relocate_object+1560>:   add    0x10(%rbx),%rax
=> 0x7ffff7de36bc <_dl_relocate_object+1564>:   mov    %rax,(%r10)
=> 0x7ffff7de36bf <_dl_relocate_object+1567>:   nop

Reading in the middle, how one could find PID of the process that one debugs in gdb? — Bulat M., Oct 02 '16 at 13:58
One can open other console (terminal) and search `ps` output for process. Also `pidof main` if the test program is called `main` and there is only one process of the program runs. — osgx, Oct 02 '16 at 18:35
Bulat, can you please remove your comment about randomize va space from http://stackoverflow.com/questions/5130654/5130690#comment66925036_5130690 and move the question here? It can be disabled in your kernel at build time (for MMU-less kernels.. or too old kernels.. or by some third-party patch); /proc can be not mounted; value can be accessed with sysctl - `sysctl -a|grep randomize`; you cat also try `setarch x86_64 -R ./program` to disable it (http://askubuntu.com/questions/318315). — osgx, Oct 02 '16 at 18:40
What do you mean by "move the question here", how to do it? I think question relevant and up to the point, it could we answered at that place. Indeed, I have MMU and relatively recent kernel with /proc mounted. — Bulat M., Oct 03 '16 at 03:34
Do you have '/proc/sys/kernel/' directory? Can you post your kernel version and list `/proc/sys/kernel/*rand*` files? Is there any randomize in `sysctl -a | grep randomize` output? — osgx, Oct 03 '16 at 03:58
Rechecked, /proc/sys/*rand* files in place. Deleted comment. — Bulat M., Oct 03 '16 at 13:43

How shared library finds GOT section?

1 Answers1