What are all the atomic operations in CUDA?

Question

I was wondering if there is a complete list of atomic operations usable in CUDA kernels. I couldn't find something like that on the internet.

Answer 1

See the CUDA Programming Guide section on atomic functions.

Answer 2

As of April 2020 (i.e. CUDA 10.2, Turing michroarchitecture), these are:

• addition
• subtraction
• minimum
• maximum
• bitwise-and
• bitwise-or
• bitwise-xor
• increment (with a wraparound value)
• decrement (with a wraparound value)
• compare-and-swap - which is perhaps the most significant, as you can "implement" essentially any atomic operation using compare-and-swap.

Note, however, that:

• Only certain data types are directly supported (of size never above 8 bytes).
• Earlier micro-architectures support less operations and/or less types.
• CUDA memory only supports aligned accesses - whether they be regular or atomic.

For details, consult the Atomic Functions section of the CUDA Programming guide.