3

I'm writing several bpf programs that will update the same counter which is implemented by a BPF_MAP_TYPE_ARRAY bpf map. To avoid data races, I referred the bpf kernel doc and used the __sync_fetch_and_add() intrinsic. However, I'm confused about how __sync_fetch_and_add() works in bpf. I found in another question that __sync_fetch_and_add() leverages atomic CPU instructions to implement atomic update, but ebpf source code written in C will be compiled into bpf bytecode, will these native atomic instructions be preserved in the bpf bytecode program after compilation? Or does the bpf instruction set also provide atomic instructions?

Thank you for your help :)

sk_buff
  • 81
  • 7

1 Answers1

3

Yes, all of that is correct. The intrinsic will emit atomic instructions in eBPF byte code. The kernels docs have the following to say about that:

Atomic operations are operations that operate on memory and can not be interrupted or corrupted by other access to the same memory region by other eBPF programs or means outside of this specification.

All atomic operations supported by eBPF are encoded as store operations that use the BPF_ATOMIC mode modifier as follows:

  • BPF_ATOMIC | BPF_W | BPF_STX for 32-bit operations
  • BPF_ATOMIC | BPF_DW | BPF_STX for 64-bit operations

8-bit and 16-bit wide atomic operations are not supported.

The ‘imm’ field is used to encode the actual atomic operation. Simple atomic operation use a subset of the values defined to encode arithmetic operations in the ‘imm’ field to encode the atomic operation:

imm value description
BPF_ADD 0x00 atomic add
BPF_OR 0x40 atomic or
BPF_AND 0x50 atomic and
BPF_XOR 0xa0 atomic xor

BPF_ATOMIC | BPF_W | BPF_STX with ‘imm’ = BPF_ADD means:

*(u32 *)(dst + offset) += src

BPF_ATOMIC | BPF_DW | BPF_STX with ‘imm’ = BPF ADD means:

*(u64 *)(dst + offset) += src

In addition to the simple atomic operations, there also is a modifier and two complex atomic operations:

imm value description
BPF_FETCH 0x01 modifier: return old value
BPF_XCHG 0xe0 | BPF_FETCH atomic exchange
BPF_CMPXCHG 0xf0 | BPF_FETCH atomic compare and exchange

The BPF_FETCH modifier is optional for simple atomic operations, and always set for the complex atomic operations. If the BPF_FETCH flag is set, then the operation also overwrites src with the value that was in memory before it was modified.

The BPF_XCHG operation atomically exchanges src with the value addressed by dst + offset.

The BPF_CMPXCHG operation atomically compares the value addressed by dst + offset with R0. If they match, the value addressed by dst + offset is replaced with src. In either case, the value that was at dst + offset before the operation is zero-extended and loaded back to R0.

These are translated into native atomic instructions during the JIT process. Here are a few links to examples of this happening for different architectures:

Dylan Reimerink
  • 5,874
  • 2
  • 15
  • 21