How to get CUDA parallel threads to write an array serially?

Question

We have multiple threads calculating results but needing to write the results in one array serially. We've tried using atomicCAS in the example below. In some parts of the code it works, in other parts of the code it hangs because of warp divergence. It doesn't matter which order the threads write the results, but they should never try to write the array at the same time.

while (atomicCAS(&arrayAccess, AVAILABLE, NOT_AVAILABLE) == NOT_AVAILABLE);

arrayGlobalMemory[count] = result;
count++;

atomicExch(&arrayAccess, AVAILABLE);

This answers below says that it isn't possible. It's pretty basic functionality. It seems that parallel access to an array should be serializable? Can someone suggest how to modify the code to get serialized array access from parallel threads, or can someone show some sample code that works correctly?

CUDA, mutex and atomicCAS()

Serially thread-wise. No 2 or more threads should try to write to the same element in the array. — roger1994, Dec 10 '15 at 02:51
The threads may all be in one block. But there also cases where there are threads across multiple blocks that need to access the array serially. — roger1994, Dec 10 '15 at 03:07

score 3 · Accepted Answer · edited Dec 10 '15 at 05:42

3

It sounds like what you really want is

unsigned int *arrayGlobalMemory; unsigned int count = 0;

...

int idx = atomicAdd(&count, 1);
arrayGlobalMemory[idx] = result;

i.e. the atomic operation isn't performed on the array, but on the array index. This prevents any two threads from writing to same location in arrayGlobalMemory.

edited Dec 10 '15 at 05:42

user703016

37,307
8
87
112

answered Dec 10 '15 at 03:09

talonmies

70,661
34
192
269

Yeah, that's what I needed. I checked it and it worked. Thanks again talonmies! – roger1994 Dec 10 '15 at 03:24
I think it needs "volatile" keyword for "count" to avoid caching – Patric Dec 10 '15 at 05:17

How to get CUDA parallel threads to write an array serially?

1 Answers1