How can an ebpf program change kernel execution flow or call kernel functions?

Question

I'm trying to figure out how an ebpf program can change the outcome of a function (not a syscall, in my case) in kernel space. I've found numerous articles and blog posts about how ebpf turns the kernel into a programmable kernel, but it seems like every example is just read-only tracing and collecting statistics.

I can think of a few ways of doing this: 1) make a kernel application read memory from an ebpf program, 2) make ebpf change the return value of a function, 3) allow an ebpf program to call kernel functions.

The first approach does not seem like a good idea. The second would be enough, but as far as I understand it's not easy. This question says syscalls are read-only. This bcc document says it is possible but the function needs to be whitelisted in the kernel. This makes me think that the whitelist is fixed and can only be changed by recompiling the kernel, is this correct? The third seems to be the most flexible one, and this blog post encouraged me to look into it. This is the one I'm going for.

I started with a brand new 5.15 kernel, which should have this functionality As the blog post says, I did something no one should do (security is not an issue since I'm just toying with this) and opened every function to ebpf by adding this to net/core/filter.c (which I'm not sure is the correct place to do so):

static bool accept_the_world(int off, int size,
    enum bpf_access_type type,
    const struct bpf_prog *prog,
    struct bpf_insn_access_aux *info)
{
    return true;
}

bool export_the_world(u32 kfunc_id) 
{
    return true;
}

const struct bpf_verifier_ops all_verifier_ops = {
    .check_kfunc_call   = export_the_world,
    .is_valid_access    = accept_the_world,
};

How does the kernel know of the existence of this struct? I don't know. None of the other bpf_verifier_ops declared are used anywhere else, so it doesn't seem like there is a register_bpf_ops

Next I was able to install bcc (after a long fight due to many broken installation guides). I had to checkout v0.24 of bcc. I read somewhere that pahole is required when compiling the kernel, so I updated mine to v1.19.

My python file is super simple, I just copied the vfs example from bcc and simplified it:

bpf_text_kfunc = """
extern void hello_test_kfunc(void) __attribute__((section(".ksyms")));
KFUNC_PROBE(vfs_open)
{ 
    stats_increment(S_OPEN); 
    hello_test_kfunc(); 
    return 0; 
}
"""

b = BPF(text=bpf_text_kfunc)

Where hello_test_kfunc is just a function that does a printk, inserted as a module into the kernel (it is present in kallsyms). When I try to run it, I get:

/virtual/main.c:25:5: error: cannot call non-static helper function
    hello_test_kfunc();
    ^

And this is where I'm stuck. It seems like it's the JIT that is not allowing this, but who exactly is causing this issue? BCC, libbpf or something else? Do I need to manually write bpf code to call kernel functions?

Does anyone have an example with code of what the lwn blog post I linked talks about actually working?

Answer 1

eBPF is fundamentally made to extend kernel functionality in very specific limited ways. Essentially a very advanced plugin system. One of the main design principles of the eBPF is that a program is not allowed to break the kernel. Therefor it is not possible to change to outcome of arbitrary kernel functions.

The kernel has facilities to call a eBPF program at any time the kernel wants and then use the return value or side effects from helper calls to effect something. The key here is that the kernel always knows it is doing this.

One sort of exception is the BPF_PROG_TYPE_STRUCT_OPS program type which can be used to replace function pointers in whitelisted structures. But again, explicitly allowed by the kernel.

1. make a kernel application read memory from an ebpf program

This is not possible since the memory of an eBPF program is ephemaral, but you could define your own custom eBPF program type and pass in some memory to be modified to the eBPF program via a custom context type.

2. make ebpf change the return value of a function

Not possible unless you explicitly call a eBPF program from that function.

3. allow an ebpf program to call kernel functions.

While possible for a number for purposes, this typically doesn't give you the ability to change return values of arbitrary functions.

You are correct, certain program types are allowed to call some kernel functions. But these are again whitelisted as you discovered.

How does the kernel know of the existence of this struct?

Macro magic. The verifier builds a list of these structs. But only if the program type exists in the list of program types.

/virtual/main.c:25:5: error: cannot call non-static helper function

This seems to be a limitation of BCC, so if you want to play with this stuff you will likely have to manually compile your eBPF program and load it with libbpf or cilium/ebpf.