GDB does not step-into functions without step-mode=on

Question

I have an executable that is dynamically linked with a .so file, both compiled with debug symbols.

Now when I start the process with a gdb and step over it - the step command does not step into functions (defined in a .so) unless I set the step-mode on.

Sources are available and gdb certainly can find them.

Why is it happening?

Here is the corresponding debug output enabled via set debug infrun 1:

34          assert_se(calendar_spec_from_string(input, &c) >= 0);
infrun: infrun_async(0)
(gdb) s
infrun: clear_proceed_status_thread (Thread 0x7ffff7fb3900 (LWP 7009))
infrun: proceed (addr=0xffffffffffffffff, signal=GDB_SIGNAL_DEFAULT)
infrun: proceed: resuming Thread 0x7ffff7fb3900 (LWP 7009)
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e0d
infrun: infrun_async(1)
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e11
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e11
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e15
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e15
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e18
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e18
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e1b
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e1b
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000c08
infrun: stepped into subroutine
infrun: inserting step-resume breakpoint at 0x100000e20
infrun: resume (step=0, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000c08
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e20
infrun: BPSTAT_WHAT_STEP_RESUME
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e20
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e23
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e23
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e26
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e26
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e29
infrun: stepping inside range [0x100000e0d-0x100000e4f]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e29
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e4f
infrun: stepped to a different line
infrun: stop_waiting
infrun: clear_step_over_info
infrun: stop_all_threads
infrun: stop_all_threads, pass=0, iterations=0
infrun:   Thread 0x7ffff7fb3900 (LWP 7009) not executing
infrun: stop_all_threads, pass=1, iterations=1
infrun:   Thread 0x7ffff7fb3900 (LWP 7009) not executing
infrun: stop_all_threads done
36          assert_se(calendar_spec_to_string(c, &p) >= 0);
infrun: infrun_async(0)
(gdb) s
infrun: clear_proceed_status_thread (Thread 0x7ffff7fb3900 (LWP 7009))
infrun: proceed (addr=0xffffffffffffffff, signal=GDB_SIGNAL_DEFAULT)
infrun: proceed: resuming Thread 0x7ffff7fb3900 (LWP 7009)
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e4f
infrun: infrun_async(1)
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e53
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e53
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e57
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e57
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e5a
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e5a
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e5d
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e5d
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000c48
infrun: stepped into subroutine
infrun: inserting step-resume breakpoint at 0x100000e62
infrun: resume (step=0, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000c48
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e62
infrun: BPSTAT_WHAT_STEP_RESUME
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e62
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e65
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e65
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e68
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e68
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e6b
infrun: stepping inside range [0x100000e4f-0x100000e91]
infrun: resume (step=1, signal=GDB_SIGNAL_0), trap_expected=0, current thread [Thread 0x7ffff7fb3900 (LWP 7009)] at 0x100000e6b
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   -1.0.0 [process -1],
infrun:   status->kind = ignore
infrun: TARGET_WAITKIND_IGNORE
infrun: prepare_to_wait
infrun: target_wait (-1.0.0, status) =
infrun:   7009.7009.0 [Thread 0x7ffff7fb3900 (LWP 7009)],
infrun:   status->kind = stopped, signal = GDB_SIGNAL_TRAP
infrun: TARGET_WAITKIND_STOPPED
infrun: stop_pc = 0x100000e91
infrun: stepped to a different line
infrun: stop_waiting
infrun: clear_step_over_info
infrun: stop_all_threads
infrun: stop_all_threads, pass=0, iterations=0
infrun:   Thread 0x7ffff7fb3900 (LWP 7009) not executing
infrun: stop_all_threads, pass=1, iterations=1
infrun:   Thread 0x7ffff7fb3900 (LWP 7009) not executing
infrun: stop_all_threads done
37          printf("\"%s\" → \"%s\"\n", input, p);
infrun: infrun_async(0)

Output from `gdb --version` and `gcc --version` may be helpful. So could be the output from `set debug infrun 1`. — Employed Russian, Sep 07 '17 at 13:42
@EmployedRussian it's 7.11.1 and 5.4.0 correspondingly (it's a ubuntu xenial). I will do `set debug infrun 1` and debug it when I have access to that machine and provide the results. — zerkms, Sep 07 '17 at 19:58
@EmployedRussian I posted the debug output for 2 steps each of which should have stepped inside the function. Sorry for posting it so late. — zerkms, Sep 08 '17 at 20:21
Can you step into `printf` at line 37? (Assuming glibc debug symbols are installed) — ks1322, Sep 13 '17 at 18:21
@ks1322 they are installed but I'm not sure what I need to do additionally: it tries to load sources from `/build/glibc-bfm8X4/glibc-2.23/stdio-common/printf.c` — zerkms, Sep 13 '17 at 21:37

score 1 · Answer 1 · answered Sep 11 '17 at 17:29

1

Seems to be related to "-Wl,-z,now" option to the linker.

man page says:

now - When generating an executable or shared library, mark it to tell the dynamic linker to resolve all symbols when the program is started, or when the shared library is linked to using dlopen, instead of deferring function call resolution to the point when the function is first called.

When you set step-mode on and gebug an executable linked with this option, you may notice that you don't step directly to the function in the shared library, but at first to somewhere like

(gdb) s
0x0000555555554638 in ?? ()

Can't say exactly, what it is.

answered Sep 11 '17 at 17:29

myokaski

61
5

Awarding a bounty because there are no better candidates :-) – zerkms Sep 16 '17 at 21:52
Thanks) But you didn't say anything if it was helpful. Did you try to turn off -Wl,-z,now flags? – myokaski Sep 20 '17 at 13:43
It's a part of a rather complicated build system (it's a `systemd` project) I'm not sure at this point I may spend time figuring out how to change a compile flag in a multi-thousand-line build config :-) – zerkms Sep 20 '17 at 20:24

GDB does not step-into functions without step-mode=on

1 Answers1