2

I am using ebpf+XDP to make some demo.

when I use a large memory MAP, for example:

BPF_HASH(cache, u64, u64, 10240000);
BPF_HASH(filter1, u32, u64, 10240000);
BPF_HASH(filter2, struct XXX, u16, 10240000);

when I run this demo, After running for a while, the program is automatically killed.

here is the error said:

Out of memory: KIll process 1618 (sshd) score 0 or sacrifice child
Killed process 1618 (sshd) total-vm:625792kB, anon-rss:0kB, file-rss:4kB, shmem-rss:0kB

I do not understand what dose this error mean.

Is this the system limit or bpf vm limit or map limit ?

Here is the result when I run "free -g".

              total        used        free      shared  buff/cache   available
Mem:              3           0           3           0           0           3
Swap:             3           0           3
Vector
  • 67
  • 1
  • 5
  • I'm unable to reproduce. Could you post an actual, minimal reproduction case? Also, what version of bcc are you using? – pchaigno Jan 17 '20 at 10:10

1 Answers1

2

When you create BPF maps, memory is allocated in kernel space for those maps. Obviously, the bigger they are, the more memory is needed.

While BPF has virtually no limitation on the size of the maps (other than the maximum value for the 32 bit unsigned integer which is passed to the kernel to give map size), the kernel itself has constraints due to the underlying hardware. If the kernel were to run completely out of memory, Bad Things Would Happen. Typically, it would hang or crash. To avoid that, when memory becomes scarce, the Out Of Memory (OOM) killer goes into action and kills a process, as an attempt to free memory.

From what I understand, this is what happens in your case. The limit is not from BPF or BPF maps, but simply from your system (kernel) running out of memory. You can maybe obtain more information in kernel logs with dmesg when the kill occurs (see also how to read the logs).

So to answer to the question as you phrased in the title: BPF maps are not limited in size, other than by the maximum value we can possibly pass to the bpf() system call (a uint_32, so that's about 4GB). Non-root users may have additional restrictions on the amount of space they may be able to lock in the kernel (root can remove those restrictions with ulimit -l in the shell or setrlimit() in a C program). Other “limits” for the BPF infrastructure exist (stack size, number of tail calls, etc.) but I do not think listing all of them here would help much. You can find more information about them in the Cilium guide.

Qeole
  • 8,284
  • 1
  • 24
  • 52
  • Hi @Qeole, in kernel source [here](https://elixir.bootlin.com/linux/v5.6/source/kernel/bpf/syscall.c#L373), it asserts that the map is too big if `size >= U32_MAX - PAGE_SIZE`, do you know the reason that there has a `- PAGE_SIZE` inside the statement? That is, is there any reason that the size should not exceed `4GB - 4KB` (say the arch is amd64) instead of simply 4GB of memory? Thanks! – Feng. Ma Jun 14 '21 at 08:20
  • 1
    Right I missed that check at the time when I answered. Before this check was moved to syscall.c, it [used to be accompanied with a comment](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/bpf/arraymap.c?id=a10423b87a7eae75da79ce80a8d9475047a674ee#n73): `/* make sure there is no u32 overflow later in round_up() */`. So I understand that the final, actual size is rounded up from the value passed by the user to make sure it is a multiple of `PAGE_SIZE`. Therefore we need to make sure that this rounding up will not cause an overflow. – Qeole Jun 14 '21 at 09:00