gcc debug symbols (-g flag) vs linker's -rdynamic option

Question

glibc provides backtrace() and backtrace_symbols() to get the stack trace of a running program. But for this to work the program has to be built with linker's -rdynamic flag.

What is the difference between -g flag passed to gcc vs linker's -rdynamic flag ? For a sample code I did readelf to compare the outputs. -rdynamic seems to produce more info under Symbol table '.dynsym' But I am not quite sure what the additional info is.

Even if I strip a program binary built using -rdynamic, backtrace_symbols() continue to work.

When strip removes all the symbols from the binary why is it leaving behind whatever was added by the -rdynamic flag ?

Edit: Follow-up questions based on Mat's response below..

For the same sample code you took this is the difference I see with -g & -rdynamic

without any option..

    Symbol table '.dynsym' contains 4 entries:
       Num:    Value          Size Type    Bind   Vis      Ndx Name
         0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
         1: 0000000000000000   218 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2.5 (2)
         2: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
         3: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__

    Symbol table '.symtab' contains 70 entries:
       Num:    Value          Size Type    Bind   Vis      Ndx Name
         0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
         1: 0000000000400200     0 SECTION LOCAL  DEFAULT    1 
         2: 000000000040021c     0 SECTION LOCAL  DEFAULT    2

with -g there are more sections, more entries in .symtab table but .dynsym remains the same..

      [26] .debug_aranges    PROGBITS         0000000000000000  0000095c
           0000000000000030  0000000000000000           0     0     1
      [27] .debug_pubnames   PROGBITS         0000000000000000  0000098c
           0000000000000023  0000000000000000           0     0     1
      [28] .debug_info       PROGBITS         0000000000000000  000009af
           00000000000000a9  0000000000000000           0     0     1
      [29] .debug_abbrev     PROGBITS         0000000000000000  00000a58
           0000000000000047  0000000000000000           0     0     1
      [30] .debug_line       PROGBITS         0000000000000000  00000a9f
           0000000000000038  0000000000000000           0     0     1
      [31] .debug_frame      PROGBITS         0000000000000000  00000ad8
           0000000000000058  0000000000000000           0     0     8
      [32] .debug_loc        PROGBITS         0000000000000000  00000b30
           0000000000000098  0000000000000000           0     0     1

    Symbol table '.dynsym' contains 4 entries:
       Num:    Value          Size Type    Bind   Vis      Ndx Name
         0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
         1: 0000000000000000   218 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2.5 (2)
         2: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
         3: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__

    Symbol table '.symtab' contains 77 entries:
       Num:    Value          Size Type    Bind   Vis      Ndx Name
         0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
         1: 0000000000400200     0 SECTION LOCAL  DEFAULT    1

with -rdynamic no additional debug sections, .symtab entries are 70 (same as vanilla gcc invocation), but more .dynsym entries..

    Symbol table '.dynsym' contains 19 entries:
       Num:    Value          Size Type    Bind   Vis      Ndx Name
         0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
         1: 0000000000000000   218 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2.5 (2)
         2: 00000000005008e8     0 OBJECT  GLOBAL DEFAULT  ABS _DYNAMIC
         3: 0000000000400750    57 FUNC    GLOBAL DEFAULT   12 __libc_csu_fini   
         4: 00000000004005e0     0 FUNC    GLOBAL DEFAULT   10 _init
         5: 0000000000400620     0 FUNC    GLOBAL DEFAULT   12 _start
         6: 00000000004006f0    86 FUNC    GLOBAL DEFAULT   12 __libc_csu_init   
         7: 0000000000500ab8     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start
         8: 00000000004006de    16 FUNC    GLOBAL DEFAULT   12 main
         9: 0000000000500aa0     0 NOTYPE  WEAK   DEFAULT   23 data_start
        10: 00000000004007c8     0 FUNC    GLOBAL DEFAULT   13 _fini
        11: 00000000004006d8     6 FUNC    GLOBAL DEFAULT   12 foo
        12: 0000000000500ab8     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
        13: 0000000000500a80     0 OBJECT  GLOBAL DEFAULT  ABS _GLOBAL_OFFSET_TABLE_
        14: 0000000000500ac0     0 NOTYPE  GLOBAL DEFAULT  ABS _end
        15: 00000000004007d8     4 OBJECT  GLOBAL DEFAULT   14 _IO_stdin_used
        16: 0000000000500aa0     0 NOTYPE  GLOBAL DEFAULT   23 __data_start
        17: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
        18: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__    

    Symbol table '.symtab' contains 70 entries:
       Num:    Value          Size Type    Bind   Vis      Ndx Name
         0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
         1: 0000000000400200     0 SECTION LOCAL  DEFAULT    1 
         2: 000000000040021c     0 SECTION LOCAL  DEFAULT    2

Now these are the questions I have..

In gdb you can do bt to get the bactrace. If that works with just -g why do we need -rdynamic for backtrace_symbols to work ?
Comparing the additions to .symtab with -g & additions to .dynsym with -rdynamic they are not exactly the same.. Does either one provide better debugging info compared to the other ? FWIW, size of the output produced is like this: with -g > with -rdynamic > with neither option
What exactly is the usage of .dynsym ? Is it all the symbols exported by this binary ? In that case why is foo going into .dynsym because we are not compiling the code as a library.
If I link my code using all static libraries then -rdynamic is not needed for backtrace_symbols to work ?

It so happens that backtrace unfortunately does not use debug symbols [if available] like most other tools, including gdb, valgrind, etc. — jørgensen, Dec 24 '11 at 12:38

score 42 · Accepted Answer · edited Aug 09 '22 at 04:35

According to the docs:

This instructs the linker to add all symbols, not only used ones, to the dynamic symbol table.

Those are not debug symbols, they are dynamic linker symbols. Those are not removed by strip since it would (in most cases) break the executable - they are used by the runtime linker to do the final link stage of your executable.

Example:

$ cat t.c
void foo() {}
int main() { foo(); return 0; }

Compile and link without -rdynamic (and no optimizations, obviously)

$ gcc -O0 -o t t.c
$ readelf -s t

Symbol table '.dynsym' contains 3 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2.5 (2)
     2: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__

Symbol table '.symtab' contains 50 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000400270     0 SECTION LOCAL  DEFAULT    1 
....
    27: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS t.c
    28: 0000000000600e14     0 NOTYPE  LOCAL  DEFAULT   18 __init_array_end
    29: 0000000000600e40     0 OBJECT  LOCAL  DEFAULT   21 _DYNAMIC

So the executable has a .symtab with everything. But notice that .dynsym doesn't mention foo at all - it has the bare essentials in there. This is not enough information for backtrace_symbols to work. It relies on the information present in that section to match code addresses with function names.

Now compile with -rdynamic:

$ gcc -O0 -o t t.c -rdynamic
$ readelf -s t

Symbol table '.dynsym' contains 17 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2.5 (2)
     2: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
     3: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
     4: 0000000000601018     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
     5: 0000000000601008     0 NOTYPE  GLOBAL DEFAULT   24 __data_start
     6: 0000000000400734     6 FUNC    GLOBAL DEFAULT   13 foo
     7: 0000000000601028     0 NOTYPE  GLOBAL DEFAULT  ABS _end
     8: 0000000000601008     0 NOTYPE  WEAK   DEFAULT   24 data_start
     9: 0000000000400838     4 OBJECT  GLOBAL DEFAULT   15 _IO_stdin_used
    10: 0000000000400750   136 FUNC    GLOBAL DEFAULT   13 __libc_csu_init
    11: 0000000000400650     0 FUNC    GLOBAL DEFAULT   13 _start
    12: 0000000000601018     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start
    13: 000000000040073a    16 FUNC    GLOBAL DEFAULT   13 main
    14: 0000000000400618     0 FUNC    GLOBAL DEFAULT   11 _init
    15: 00000000004007e0     2 FUNC    GLOBAL DEFAULT   13 __libc_csu_fini
    16: 0000000000400828     0 FUNC    GLOBAL DEFAULT   14 _fini

Symbol table '.symtab' contains 50 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000400270     0 SECTION LOCAL  DEFAULT    1 
....
    27: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS t.c
    28: 0000000000600e14     0 NOTYPE  LOCAL  DEFAULT   18 __init_array_end
    29: 0000000000600e40     0 OBJECT  LOCAL  DEFAULT   21 _DYNAMIC

Same thing for symbols in .symtab, but now foo has a symbol in the dynamic symbol section (and a bunch of other symbols appear there now too). This makes backtrace_symbols work - it now has enough information (in most cases) to map code addresses with function names.

Strip that:

$ strip --strip-all t
$ readelf -s t

Symbol table '.dynsym' contains 17 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND 
     1: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND __libc_start_main@GLIBC_2.2.5 (2)
     2: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND __gmon_start__
     3: 0000000000000000     0 NOTYPE  WEAK   DEFAULT  UND _Jv_RegisterClasses
     4: 0000000000601018     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
     5: 0000000000601008     0 NOTYPE  GLOBAL DEFAULT   24 __data_start
     6: 0000000000400734     6 FUNC    GLOBAL DEFAULT   13 foo
     7: 0000000000601028     0 NOTYPE  GLOBAL DEFAULT  ABS _end
     8: 0000000000601008     0 NOTYPE  WEAK   DEFAULT   24 data_start
     9: 0000000000400838     4 OBJECT  GLOBAL DEFAULT   15 _IO_stdin_used
    10: 0000000000400750   136 FUNC    GLOBAL DEFAULT   13 __libc_csu_init
    11: 0000000000400650     0 FUNC    GLOBAL DEFAULT   13 _start
    12: 0000000000601018     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start
    13: 000000000040073a    16 FUNC    GLOBAL DEFAULT   13 main
    14: 0000000000400618     0 FUNC    GLOBAL DEFAULT   11 _init
    15: 00000000004007e0     2 FUNC    GLOBAL DEFAULT   13 __libc_csu_fini
    16: 0000000000400828     0 FUNC    GLOBAL DEFAULT   14 _fini
$ ./t
$

Now .symtab is gone, but the dynamic symbol table is still there, and the executable runs. So backtrace_symbols still works too.

Strip the dynamic symbol table:

$ strip -R .dynsym t
$ ./t
./t: relocation error: ./t: symbol , version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference

... and you get a broken executable.

An interesting read for what .symtab and .dynsym are used for is here: Inside ELF Symbol Tables. One of the things to note is that .symtab is not needed at runtime, so it is discarded by the loader. That section does not remain in the process's memory. .dynsym, on the otherhand, is needed at runtime, so it is kept in the process image. So it is available for things like backtrace_symbols to gather information about the current process from within itself.

So in short:

dynamic symbols are not stripped by strip since that would render the executable non-loadable
backtrace_symbols needs dynamic symbols to figure out what code belongs which function
backtrace_symbols does not use debugging symbols

Hence the behavior you noticed.

For your specific questions:

gdb is a debugger. It uses debug information in the executable and libraries to display relevant information. It is much more complex than backtrace_symbols, and inspects the actual files on your drive in addition to the live process. backtrace_symbols does not, it is entirely in-process - so it cannot access sections that are not loaded into the executable image. Debug sections are not loaded into the runtime image, so it can't use them.
.dynsym is not a debugging section. It is a section used by the dynamic linker. .symbtab isn't a debugging section either, but it can be used by debugger that have access to the executable (and library) files. -rdynamic does not generate debug sections, only that extended dynamic symbol table. The executable growth from -rdynamic depends entirely on the number of symbols in that executable (and alignment/padding considerations). It should be considerably less than -g.
Except for statically linked binaries, executables need external dependencies resolved at load time. Like linking printf and some application startup procedures from the C library. These external symbols must be indicated somewhere in the executable: this is what .dynsym is used for, and this is why the exe has a .dynsym even if you don't specify -rdynamic. When you do specify it, the linker adds other symbols that are not necessary for the process to work, but can be used by things like backtrace_symbols.
backtrace_symbols will not resolve any function names if you statically link. Even if you specify -rdynamic, the .dynsym section will not be emitted to the executable. No symbol tables gets loaded into the executable image, so backtrace_symbols cannot map code adresses to symbols.

thanks for the response.. but I am still pretty confused.. Let me enter my questions as a separate response so that it can be better formatted. — Manohar, Dec 24 '11 at 19:28
Do **not** post an answer as a followup, that is not how this site works. Edit your original question (or post a new one if your new question is sufficiently different). — Mat, Dec 24 '11 at 19:29
@Santhosh: added some info at the end, and tried to clarify the whole thing. HTH. — Mat, Dec 24 '11 at 19:49
Thanks, Mat.. Just a moment back I edited my question with additional info. — Manohar, Dec 24 '11 at 20:06
@Santhosh: updated. Please also read that link, and don't confuse debug information with information that is required for the program to link/load and run. — Mat, Dec 24 '11 at 20:24
the link you sent explains exactly what I was looking for.. .symtab is a super set of .dynsym. so gdb can work with .symtab. I guess it should have been possible to write backtrace_symbols to work .symtab. But since the size of the binary is much higher with .symtab (built with -g) the designers might have opted for using .dynsym instead. — Manohar, Dec 24 '11 at 20:25
**`.symtab` is not loaded into the process image**. It is not available for `backtrace_symbols` to use. — Mat, Dec 24 '11 at 20:27
Why is the value of `_gmon_start` set to 0x0? What happens when the `PLT` pushes the entry for `_gmon_start` onto the stack and calls the dynamic linker? It seems like the executable would have no way of knowing where `_gmon_start` is. — RouteMapper, Jul 19 '13 at 15:05
See http://stackoverflow.com/questions/12697081/what-is-gmon-start-symbol, that's just some sort of marker. It'll get a value if the program is compiled with pgo enabled. Not sure why it needs to be there when profiling isn't enabled. — Mat, Jul 19 '13 at 15:43

gcc debug symbols (-g flag) vs linker's -rdynamic option

1 Answers1

Linked