How can I get the lua stack trace from a core file using gdb

Question

I have a C++ application (for OS X) that calls lua as a scripting language. I'm running a large number of these applications (100s) and they can run for a very long time (days or weeks).

Sometimes one crashes. And when it crashes it leaves me a lovely core file.

I can open this core file in gdb and find where the application crashes. I can walk the call stack and find an instance of a lua_State variable. My problem is that I'd like to see what the lua call stack looks like at this time...

Keep in mind that since this is a core I don't have access to calling C functions, which rules out several of the usual ways of debugging lua scripts.

Id like to avoid adding manual traces through debug hooks as I'm worried about the additional performance penalties, and added complexity.

How can I traverse the lua internal structures to get at call stack information?

So, maybe you've read this already, but i don't know, so don't be grumpy, please ;) [Lua callstack with C++ debugger](http://zeuxcg.org/2010/11/07/lua-callstack-with-c-debugger/) — Sebastian, Dec 19 '11 at 07:02
@macs That's a pretty great overview. The "Inspect Lua data structures" section is the key. I'd worked out most of that but its pretty cumbersome to use. I'll probably look to writing some GDB macros/scripts to make it workable. — Michael Anderson, Dec 19 '11 at 08:04
I'm glad giving you a helping hand, there is also [StackTracePlus](https://github.com/ignacio/StackTracePlus) but you'll have to modify the calling C function, if i'm correct. So it's rather useless in this particular case. — Sebastian, Dec 19 '11 at 08:08
@macs that comment is probably the closest thing to an answer for this question atm. no sense letting a +150 bounty go to waste right? ;) — greatwolf, Dec 21 '11 at 05:39
@VictorT.: Actually I was only waiting for such a comment ;) — Sebastian, Dec 21 '11 at 15:34
The insightful article referenced above (written in 2010) is probably only totally valid for Lua 5.1. Lua 5.2 wasn't released until late 2011. — mojo, Jun 22 '20 at 18:34

Michael Anderson · Accepted Answer · 2011-12-22T07:32:12.170

I've created a GDB script to do the stuff in the web page linked to by macs. Its not beautiful, and should probably be properly wrapped into a function etc, but here it is for the curious.

NOTE: It seems that the web page is wrong about the filename for lua functions. In the case where the string comes from luaL_dofile() the filename starts with a @ symbol. If they're called from lua_dostring(). In that case the $filename variable is set to the whole of the string passed to lua_dostring() - and the user is probably only interested in one or two lines of context from that file. I wasn't sure how to fix that up.

set $p = L->base_ci
while ($p <= L->ci )
  if ( $p->func->value.gc->cl.c.isC == 1 )
    printf "0x%x   C FUNCTION", $p
    output $p->func->value.gc->cl.c.f
    printf "\n"
  else
    if ($p->func.tt==6)
      set $proto = $p->func->value.gc->cl.l.p
      set $filename = (char*)(&($proto->source->tsv) + 1)
      set $lineno = $proto->lineinfo[ $p->savedpc - $proto->code -1 ]
      printf "0x%x LUA FUNCTION : %d %s\n", $p, $lineno, $filename
    else
      printf "0x%x LUA BASE\n", $p
    end
  end
  set $p = $p+1
end

This outputs something like:

0x1002b0 LUA BASE
0x1002c8 LUA FUNCTION : 4 @a.lua
0x1002e0 LUA FUNCTION : 3 @b.lua
0x100310   C FUNCTION(lua_CFunction) 0x1fda <crash_function(lua_State*)>

When I debug the crash from this code:

// This is a file designed to crash horribly when run.
// It should generate a core, and it should crash inside some lua functions

#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"

#include <iostream>
#include <signal.h>

int crash_function(lua_State * L)
{
  raise( SIGABRT ); //This should dump core!
  return 0;
}



int main()
{
  lua_State * L = luaL_newstate();
  lua_pushcfunction(L, crash_function);
  lua_setfield(L, LUA_GLOBALSINDEX, "C");

  luaopen_base(L);
  if( 1 == luaL_dofile(L, "a.lua" ))
  {
    std::cout<<"ERROR: "<<lua_tostring(L,-1)<<std::endl;
    return 1;
  }
  if( 1 == luaL_dofile(L, "b.lua" ))
  {
    std::cout<<"ERROR: "<<lua_tostring(L,-1)<<std::endl;
    return 1;
  }

  lua_getfield(L, LUA_GLOBALSINDEX, "A");
  lua_pcall(L, 0, 0, NULL);
}

With a.lua

-- a.lua
-- just calls B, which calls C which should crash
function A()
  B()
end

and b.lua

-- b.lua
function B()
  C()
end

score 8 · Answer 2 · answered Nov 02 '12 at 04:21

This is a small variation to Michael Anderson's GDB script: I had to use this because I was getting Cannot access memory at address 0x656d errors with his script, due to L->base_ci being invalid in my core dump. This starts from the top frame (L->ci) and goes down, in the opposite direction, avoiding the invalid L->base_ci pointer.

set $p = L->ci
while ($p > L->base_ci )
  if ( $p->func->value.gc->cl.c.isC == 1 )
    printf "0x%x   C FUNCTION ", $p
    output $p->func->value.gc->cl.c.f
    printf "\n"
  else
    if ($p->func.tt==6)
      set $proto = $p->func->value.gc->cl.l.p
      set $filename = (char*)(&($proto->source->tsv) + 1)
      set $lineno = $proto->lineinfo[ $p->savedpc - $proto->code -1 ]
      printf "0x%x LUA FUNCTION : %d %s\n", $p, $lineno, $filename
    else
      printf "0x%x LUA BASE\n", $p
    end
  end
  set $p = $p - 1
end

score 5 · Answer 3 · answered Dec 21 '11 at 15:36

5

Based on the comments above, I'd recommend the following article: Lua callstack with C++ debugger. It's giving a good overview about debugging the Lua / C++ combination, especially the section "Inspect Lua data structures" is helpful, when it comes to debugging of core dumps.

answered Dec 21 '11 at 15:36

Sebastian

8,046
2
34
58

1

Awarded this the bounty as it is what let me get to my (partial) solution. – Michael Anderson Dec 23 '11 at 00:17

score 4 · Answer 4 · answered Dec 21 '11 at 16:01

4

You could check out my Lua GDB helpers. It is a collection of macros that let you inspect the stack and values, and even print backtrace. Essentially what the article referenced by macs contains, in a simple-to-use package.

It provides these macros:

luastack [L] - lists the values on the current Lua C stack.
luaprint < value > [verbose] - Pretty-prints a TValue passed as argument. Expects a pointer to a TValue. When verbose is 1, expands tables, metatables and userdata environments.
luaprinttable < table > - Pretty-prints a Lua Table. Expects a pointer to Table.
luavalue < index > [L] - Dumps a single value at an index.
luatraceback [L] - Calls debug.traceback(). Not sure if it will work on core file though...

answered Dec 21 '11 at 16:01

Michal Kottman

16,375
3
47
62

Apparently, your macros use the approach number 1, calling Lua methods to get the stack. It will not work with a core dump. – Paulo Henrique Silva Dec 21 '11 at 18:12
Hmmm, looks like I will have to implement more functionality in pure GDB :) – Michal Kottman Dec 21 '11 at 19:21
After taking a long and thorough look at the Lua sources, I give up implementing the stack trace in GDB, because for example to get the function name, [you have to inspect the bytecode](http://www.lua.org/source/5.1/ldebug.c.html#symbexec). I am not sure it is really possible to do offline. – Michal Kottman Dec 21 '11 at 22:06

How can I get the lua stack trace from a core file using gdb

4 Answers4

Linked