0

I'm trying to debug a core dump of a large project with gdb. The problem is that the application is a set of quite a lot of shared objects. Also each component is encrypted. Now it is quite a lot of work to prepare whole environment locally (might take even couple of days). So I was hoping that when I try to debug the coredump I will at least get .so names in backtrace, so I can fetch only the files which I really need. Unfortunately when doing this I just get empty results like this:

Thread 3 (LWP 4582):
#0  0x00007f11956d6d50 in ?? ()
#1  0x00007f112a810a68 in ?? ()
#2  0x000000002a810ab8 in ?? ()
#3  0x00007f1134001b00 in ?? ()
#4  0x00007f112a810a90 in ?? ()
#5  0x00007f112a810a68 in ?? ()
#6  0x00007f1169d3ba5b in ?? ()
#7  0x000000000163ccd8 in ?? ()
#8  0x0000000000000000 in ?? ()

Is it correct that coredump doesn't even know the name of executable nor shared object which caused an error? Is there any way to get this information?

Thank you

Lukasz Kamisinski
  • 1
  • What does `(gdb) info shared` output? – Employed Russian Oct 01 '21 at 14:53
  • `(gdb) info shared No shared libraries loaded at this time.` But this is expected since there are no yet any libraries. By using gdb this way I wanted to determine which libraries I need. – Lukasz Kamisinski Oct 01 '21 at 16:05
  • "But this is expected since there are no yet any libraries." -- no, it's not. GDB should show which libraries were loaded at crash time. Also, you are asking "how do I print libraries in the backtrace". How would GDB print that if it doesn't know which libraries were used? – Employed Russian Oct 01 '21 at 23:40
  • Oh, this I was not aware of. I thought it shows the libraries actually parsed by gdb on local PC. Does it means that my core dump is somehow corrupted? There should be like hundreds of .so files shown then. – Lukasz Kamisinski Oct 02 '21 at 04:38

1 Answers1

0

The most likely cause of this:

(gdb) info shared No shared libraries loaded at this time

is that you are analyzing a core from a remote machine, and are not doing it correctly.

In particular, GDB knows which shared libraries were loaded in the process at crash time by examining contents of _r_debug variable in the loader (this isn't exactly correct, but is close enough).

GDB finds the address of _r_debug by looking at local ld-linux, and it finds the value by looking up that address in the core.

If the ld-linux present on the machine at crash point is different from ld-linux present on the analysis machine, then GDB will effectively look at a random address, and will not be able to locate any shared libraries.

To set this up correctly, use this answer. At a minimum, you must provide the right ld-linux.so. While you are at it, also copy libc.so.6 and libpthread.so.0 -- these are required to get a correct stack trace if the program was terminated by a signal.

Employed Russian
  • 199,314
  • 34
  • 295
  • 362
  • Your solution makes sense. This will be hard though to get additional libraries from clients PC. Will try that out and get back on this. – Lukasz Kamisinski Oct 04 '21 at 11:56