3

I'm running a program on a remote machine that I can only interact with using stdout. I have a segfault somewhere in the program and I'm trying to figure out where. Is it possible to write a signal handler for sigsegv that gives me the line number and the file where it happened?

user3666471
  • 907
  • 11
  • 24
  • Why can't you use gdb on a remote machine to backtrace the segfault? – Igor S.K. Feb 28 '15 at 19:26
  • I don't have full access to this system. I can only submit my C files there and it will execute the binary and give me the output. – user3666471 Feb 28 '15 at 19:44
  • 1
    That will get you the *address* of the fault. Provided they're compiling your code to include debug info, you're going to have to do *a lot* of work to parse the DWARF data to determine the actual line number. It would be far easier to build this locally and debug it with traditional methods (i.e. gdb). – Jonathon Reinhart Feb 28 '15 at 19:48
  • You could try `system("addr2line --help");` in a simple program to check if it is available on this system, if yes, then write a signal handler which calls it for the instruction pointer where the signal occurred. – 4566976 Mar 01 '15 at 15:11

1 Answers1

3

That sounds like a horrible debugging environment, but if it's really infeasible to get GDB running, you could try the following (assuming Linux):

  1. Register a handler for SIGSEGV, SIGBUS, or whatever you think the fatal signal is using sigaction(), and pass sa.sa_flags = SA_SIGINFO. Use the sa_sigaction rather than sa_handler member of struct sigaction to register the handler.
  2. The handler will receive a void *context argument. Assuming X86_64 (you will have to figure out what the corresponding index is for the instruction pointer on other architectures), you can get the address where the signal was triggered via ((ucontext_t*)context)->uc_mcontext.gregs[REG_RIP].
  3. Once you have the address, run e.g. addr2line -Cfip -e <binary with debugging symbols> <address> to get the function and line number. (If you're cross-compiling, you need to use the addr2line from the toolchain, which will probably have some prefix, e.g. arm-linux-androideabi-addr2line.)

(Incidentally, I just recalled the context signal handler argument from another question. :)

A limitation to the above approach is that it probably won't give you a line number for crashes inside libraries -- especially if they're loaded at random addresses.

Another approach is to use backtrace(3), which is available in glibc and a few other libc's. Bit hackish, but you could write the addresses you get from it (as strings) to popen("addr2line -Cfip -e <binary with debugging symbols>") (/proc/self/exe could be used too on Linux) to generate a backtrace with line numbers on stdout. backtrace_symbols_fd() is worth looking into too, though it won't give you line numbers. It needs -rdynamic when compiling.

Edit:

It seems GDB uses personality(2) and ADDR_NO_RANDOMIZE to turn of address space randomization for libraries (would prolly need to be followed by a re-execve()). If you're really desperate, maybe that (or /proc/sys/kernel/randomize_va_space) could be used to get line numbers inside libraries too.

Ulfalizer
  • 4,664
  • 1
  • 21
  • 30