0

I have a core dump from a C++ application running on an embedded imx6 board (yocto linux). I can put gdb on the box and run it in a terminal to examine the core file like so just fine:

gdb myApplication core.udpsrc256:src.1520419431.5526

I get extremely limited information, and really need to know more about what caused the core dump. All I have is a printout from the application:

(myApplication:5526): GLib-ERROR **: ../../glib-2.46.2/glib/gmem.c:100: failed to allocate 65611 bytes
./run-app.sh: line 8:  5526 Trace/breakpoint trap   (core dumped) XDG_RUNTIME_DIR=/run/user/root ./myApplication

Also the core dump backtrace gives some useless stuff. I need to know more stuff up the stack that led to this frame:

#0  0x75ff1910 in raise () from /lib/libc.so.6
[Current thread is 1 (LWP 5533)]
(gdb) 
(gdb) 
(gdb) bt
#0  0x75ff1910 in raise () from /lib/libc.so.6
#1  0x6b169558 in g_logv () from /usr/lib/libglib-2.0.so.0
#2  0x6b169610 in g_log () from /usr/lib/libglib-2.0.so.0
#3  0x6b1681c4 in g_malloc () from /usr/lib/libglib-2.0.so.0
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Sidenote -- there is some warnings when I startup gdb:

GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-poky-linux-gnueabi".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from qt5qmlvideo...done.

warning: exec file is newer than core file.
[New LWP 5533]
[New LWP 5526]
[New LWP 5531]
[New LWP 5528]
[New LWP 5534]
[New LWP 21064]
[New LWP 5536]
[New LWP 21065]
[New LWP 5532]
[New LWP 5527]
[New LWP 5530]
[New LWP 5537]

warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.
Core was generated by `./qt5qmlvideo -platform wayland'.
Program terminated with signal SIGTRAP, Trace/breakpoint trap.
#0  0x75ff1910 in raise () from /lib/libc.so.6
[Current thread is 1 (LWP 5533)]
(gdb)

Can anyone help? Do I need some of the stuff gdb warns about... or can i rebuild my application and its dependencies in some other configuration that would give more output? Thank you!

Some more notes that may matter -

This is a multithreaded application running a gstreamer pipeline. Many gstreamer plugins generate their own threads, one of which in this pipeline is 'udpsrc'. I'm wondering if it's because this failure happens in one of those threads is the reason why I can't get details, but I want to know how to get it to show the details if possible!

adowdy
  • 329
  • 2
  • 16
  • 1
    _"exec file is newer than core file."_ That's a big red flag. Why/how is your executable newer than the coredump? – Lightness Races in Orbit Mar 08 '18 at 00:37
  • That's a keen observation. It says that because I recopied the application over to the target device, but the application was not changed in any way (so far as I know). It should be the same file with which the core was generated bit by bit – adowdy Mar 08 '18 at 01:52
  • Okay good stuff. (Usually a copy would keep the original timestamp, but this depends on your copying software.) – Lightness Races in Orbit Mar 08 '18 at 10:45

2 Answers2

0

(1) The

Do you need "set solib-search-path" or "set sysroot"?

is a problem. Check the path (on your device) where linux-vdso.so.1 resides, and include that in the solib-search-path. Similarly for the other shared-object libraries that your program uses. E.g. if some shared-object libraries are in /lib, some are in /usr/adowdy/lib and some are in /usr/adowdy/arm/lib, you can say:

(gdb) set solib-search-path /lib:/usr/adowdy/lib:/usr/adowdy/arm/lib

(2) The

warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available.

is also a problem. See the answer to this question

(3) The

failed to allocate 65611 bytes

is a clue. Are you, by any chance, trying to allocate a negative number of bytes (maybe 65536 - 65611 = -75 bytes)?

mere3ortal
  • 285
  • 2
  • 8
0

Also the core dump backtrace gives some useless stuff.

It's not entirely useless. The stack trace, and the message from the application say the same thing: your application ran out of memory (malloc failed to allocate 65611 bytes).

While a more complete stack would tell you which particular call to g_malloc failed, it's very likely to not matter in practice -- if this g_malloc didn't fail, the next one would.

You likely have a memory leak, or are simply allocating too much memory for what your system allows.

You should look into many debugging tools built for solving this exact problem.

Employed Russian
  • 199,314
  • 34
  • 295
  • 362
  • You are right. I just said that because the stack trace information gives zero new information that I don't already know from the log error. I'm hoping to use GDB to glean more about what part of the program actually causes this allocation error. It's frighteningly huge codebase to review every single spot where a malloc type call is initiated in this way. I was hoping to at least get to the point where gdb can show the thread names, for instance! – adowdy Mar 08 '18 at 03:54