I would like to enable/set the trap flag on an x86_64 machine, then execute some instructions, then disable/unset the trap flag again. When trapping, I want my own handler to be called. My attempt to do this is below.
It does not work as expected, instead, the trap seems to happen and my program is aborted. Why is that so and how can I fix it?
The output I currently see is this (on a linux 5.10.0 machine):
./test
Starting execution...
Trace/breakpoint trap
#include <assert.h>
#include <signal.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
void setTrapFlag() {
asm volatile("pushfq\n" // push status register to stack
"xorl $0x100, (%rsp)\n" // set trap-flag of on-stack value
"popfq\n" // pop status register
);
}
void trapHandler(int signo, siginfo_t *info, void *context) {
setTrapFlag();
}
int main ()
{
struct sigaction trapSa;
// set up trap signal handler
trapSa.sa_flags = SA_SIGINFO;
trapSa.sa_sigaction = trapHandler;
int ret = sigaction(SIGTRAP, &trapSa, NULL);
assert(ret == 0);
printf("Starting execution...\n");
setTrapFlag();
printf("Set trap flag!\n");
setTrapFlag();
printf("UnSet trap flag!\n");
printf ("all done\n");
return 0;
}
I was expecting an output of
./test
Starting execution...
Set trap flag!
UnSet trap flag!
all done