0

I'll try to summarize but it's gonna be complicated.

I'm having an operating system course in my university, and i have a lab work to do. I'm working in Rust (the lab work is said to be doable in any compiled language but was principally designed for C in the first time).

So I have a tracer program and a traced program.

The goal of this step of the lab work is to attach from the tracer to the traced with ptrace, then to inject "trap-call-trap" instructions at the good place to replace an existing useless function, with the function posix_memalign (from the libc) in the traced, by an indirect call via registers (with rax). The goal is to allocate memory to be able later to call a cache code from another file into the traced program.

The problem i have is that i achieve to do a posix memalign in the tracer so i know the function works, but when i call it in the traced (via the tracer), and look in register rax for the return of the function, i always get a "12" which corresponds to ENOMEM (ENOMEM Not enough space/cannot allocate memory).

I have 2 separated cargo projects to be able to launch each program separately from cargo. Everything is on this git : https://github.com/Carrybooo/TP_SEL

I'm sorry all prints and outputs are in french, (some comments too, i'm in a french course) and i didn't think i would have to share it with anyone. There is a lot of useless code too in it, from the previous steps of the lab, that i have kept just in case, so my code is not really clean.

This is the part where i attach and modify the regs to call the function (I shortened the code and didn't show you the auxiliaries functions declarations cause it would be too long) :


    ptrace::attach(pid_ptrace) //attaching to process

    wait(); //wait after attaching

    inject(pid_trace, offset_fct_to_replace, false); //injecting trap-call-trap
    ptrace::cont(pid_ptrace, Signal::SIGCONT);

    wait(); //wait for 1st trap

    let mut regs =
        ptrace::getregs(pid_ptrace);

    let ptr_to_ptr: *mut *mut c_void = ptr::null_mut();

    regs.rax = get_libc_address(pid_trace).unwrap() + get_libc_offset("posix_memalign").unwrap();
    regs.rsp = regs.rsp - (size_of::<*mut *mut c_void>() as u64);
    regs.rdi = ptr_to_ptr as u64;
    regs.rsi = size_of::<usize>() as u64;
    regs.rdx = size_of::<usize>() as u64;

    ptrace::setregs(pid_ptrace, regs); //set regs with modification

    ptrace::cont(pid_ptrace, Signal::SIGCONT);

    wait();

    let regs = ptrace::getregs(pid_ptrace);

    ptrace::detach(pid_ptrace, Signal::SIGCONT) //detaching

And running the program in the terminal gives something like this :

before modification of regs:
rax = 6
rip = 55e6c932ddc1
rsp = 7ffcee0b7fb8

before function execution:
rax = 7f59b935ded0
rdi = 0
rip = 55e6c932ddc1
rsp = 7ffcee0b7fb0

after function execution:
rax = 12                                   <------//RESULT OF THE CALL IS HERE
rdi = 55e6cac6aba0
rip = 7f59b935df20
rsp = 7ffcee0b7f90

//end of program

So yeah i don't know why i keep getting an error on this call, i presume it's because it violates rust memory safety, because the compiler never knew in the traced program that it would have to allocate memory, but i'm not sure of it, neither how to bypass it. I hope that i've been clear enough, let me know if you need any more detail, and i really thank in advance anyone who could help me. Every advice will be welcome.

Carryboo
  • 1
  • 2
  • Why `regs.rdx = 0; //size_of::() as u64;`? Don't you want to allocate something? – HHK Nov 28 '21 at 18:36
  • I've tried with many different parameters, to see if the size or alignment was the problem, here it is 0 because it's the last value i tested, but it's the same result for 4096 or anything else i've tried. I'll edit this to put back the last value. – Carryboo Nov 28 '21 at 18:49
  • I was asking because glibc `mem_align` calls `malloc` for alignments smaller or equal to the system ptr size and malloc return NULL for a 0 size, which mem_align interprets as failure to allocate (without even checking the errno) and returning ENOMEM. – HHK Nov 28 '21 at 20:36
  • 1
    Ok i understand your point. I've put the `size_of::()` back, because upper in the code (not shown in the question) i've tested the function in the local program with these parameters and it worked fine. Right now I'm using the same parameters because I know it works locally at least. – Carryboo Nov 28 '21 at 20:52

1 Answers1

0

So I went with the libc memalign instead of using posix_memalign. It's not as restricted, and it works well while calling with registers in the traced program.

Still no idea of why posix_memalign doesn't work while called from the regs in a rust program.

Carryboo
  • 1
  • 2