1

I have a coredump file from a colleague's machine.

We both have the same sources for the program and the same third party *.so files (like libmysqlclient18 and several in-house ones).

The problem is that we both compile the software (from the same sources) independently and I want to use his core dump files for inspection with GDB on my machine.

When I try to load the core file and my executable into gdb I get:

user@ubuntu:/mnt/hgfs/share/dir$ gdb prog core
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /mnt/hgfs/share/dir/prog...done.

warning: exec file is newer than core file.
[New LWP 4465]
[New LWP 4462]
[New LWP 4464]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `./prog'.
Program terminated with signal 11, Segmentation fault.
#0  0x002d3706 in ?? () from /lib/i386-linux-gnu/libc.so.6
(gdb) bt full
#0  0x002d3706 in ?? () from /lib/i386-linux-gnu/libc.so.6
No symbol table info available.
#1  0x00000000 in ?? ()
No symbol table info available.
(gdb)

Is this possible for this scenario to work? (I compile the software with debugging symbols enabled, of course) If not, what are the technical details I'm missing?

I know for sure that it wouldn't be possible if he or I would make modifications to the source, since then the executable would be different, but this is not the case, the third party *.so files and the sources, they all match.

UPDATE: After installing libc6-dbg as user mkfs suggested in the comments, I get this in gdb:

user@ubuntu:/mnt/hgfs/share/dir$ gdb prog core
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /mnt/hgfs/share/dir/prog...done.

warning: exec file is newer than core file.
[New LWP 4465]
[New LWP 4462]
[New LWP 4464]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `./prog'.
Program terminated with signal 11, Segmentation fault.
#0  0x002d3706 in _IO_helper_overflow (s=0x0, c=0) at vfprintf.c:2188
2188    vfprintf.c: No such file or directory.
(gdb) bt
#0  0x002d3706 in _IO_helper_overflow (s=0x0, c=0) at vfprintf.c:2188
#1  0x00000000 in ?? ()
(gdb) bt full
#0  0x002d3706 in _IO_helper_overflow (s=0x0, c=0) at vfprintf.c:2188
        written = 47
        target = <optimized out>
        used = -1226838776
#1  0x00000000 in ?? ()
No symbol table info available.
(gdb)
Paul
  • 20,883
  • 7
  • 57
  • 74
  • Are you sure you have the debug symbols for libc (e.g. libc6-dbg package) installed? Also, since it is looking in . for the executable, you could put your collegue's executable and core dump in a temp dir, and run gdb from there. – mkfs Sep 24 '16 at 17:01
  • This will work if the build is exactly the same including the compiler flags. If there are difference in the build env then will much easier just to share the executable. – Matthew Fisher Sep 24 '16 at 23:57
  • @MatthewFisher The flags are the same, we use the same Makefile. – Paul Sep 25 '16 at 07:22
  • @mkfs I updated my question after installing `libc6-dbg`, it seems to have helped a bit, but not fully. – Paul Sep 25 '16 at 07:23
  • It may be that the system libraries differ on the two systems, meaning you would have to copy libc etc from your colleague's computer to yours. See https://en.wikibooks.org/wiki/Linux_Applications_Debugging_Techniques/Core_files and http://stackoverflow.com/questions/7557283/how-do-i-easily-package-libraries-needed-to-analyze-a-core-dump-i-e-packcore. You should also verify that this 2-level stack trace it *not* what your colleague sees when he opens the core file in gdb : it is possible that the stack got completely hosed, and that gdb on your system is interpreting the core file correctly. – mkfs Sep 26 '16 at 12:40

1 Answers1

1

Is this possible for this scenario to work?

Yes, but you need to ensure that the symbol layout of the binary you build on both machines is identical (or at least close enough). This isn't necessarily trivial: things like local username, pathname for sources or installation directory, and hostname sometimes leak into the built object files, and may cause symbol mismatch.

To check whether the binaries are close, run diff <(nm a.out) <(nm b.out) -- there should only be few differences. If you see a lot of differences, your binaries aren't close enough.

I compile the software with debugging symbols enabled, of course

This may be your first mistake: if your coworker builds with -O2, and you build with -g (and implied -O0), the binary is guaranteed to not match.

You need to build with exactly the flags your coworker builds with (but you may add debugging symbols; e.g. if your coworker builds with -O2, you should build with -O2 -g together).

P.S. Note that you also need identical versions of system libraries on the two machines.

Community
  • 1
  • 1
Employed Russian
  • 199,314
  • 34
  • 295
  • 362