Race condition and atomic operations in Julia and other languages

Question

I have a several questions about atomic operations and multithreading.

function counter(n)
    counter = 0
    for i in 1:n
        counter += i
    end
    return counter
end

If atomic operations are used to change the global variable "counter", would that help get rid of the race condition?

Does protocol of cache coherence have any real effect to perfomance? Virtual machines like the JVM can use their own architectures to support parallel computing.
Do atomic arithmetic and similar operations require more or less resources than ordinary arithmetic?

It's difficult for me now. Hope for your help.

If atomic operations were faster, compilers would use them for everything. Atomic RMWs are quite a bit more expensive, like even with no contention for the cache line from other cores, about 20x to 100x worse throughput on modern x86 CPUs than incrementing a register. Plus being a full memory barrier, hurting out-of-order exec of loads/stores. (related: [Can num++ be atomic for 'int num'?](https://stackoverflow.com/q/39393850) explains how CPUs handle them, including getting exclusive ownership of the containing cache line). — Peter Cordes, Dec 04 '22 at 22:10
You must be in the same class as the asker of this question - https://stackoverflow.com/questions/74684854/why-does-using-an-atomic-operation-still-allow-a-race-condition. You should get together and help each other out. — High Performance Mark, Dec 05 '22 at 13:20

score 6 · Accepted Answer · answered Dec 04 '22 at 22:39

I don't quite understand your example, the variable counter seems to be local, and then there will be no race conditions in your example.

Anyway, yes, atomic operations will ensure that race conditions do not occur. There are 2 or 3 ways to do that.

1. Your counter can be an Atomic{Int}:

using .Threads
const counter = Atomic{Int}(0)
...
function updatecounter(i)
    atomic_add!(counter, i)
end

2. You can use a field in a struct declared as @atomic:

mutable struct Counter
    @atomic c::Int
end
const counter = Counter(0)
...
function updatecounter(i)
    @atomic counter.c += i
end

This is described here: https://docs.julialang.org/en/v1/base/multi-threading/#Atomic-operations It seems the details of the semantics haven't been written yet, but it's the same as in C++.

3. You can use a lock:

counter = 0
countlock = ReentrantLock()
...
function updatecounter(i)
    @lock countlock global counter += i
end

and 2. are more or less the same. The lock approach is slower, but can be used if several operations must be done serially. No matter how you do it, there will be a performance degradation relative to non-atomic arithmetic. The atomic primitives in 1. and 2. must do a memory fence to ensure the correct ordering, so cache coherence will matter, depending on the hardware.

1 Answers1