Shared read resources across threads

Question

This is a question about read performance in multi-threaded code.

So here is the situation. I have a large amount of data that a couple thousand entities need to know in order to react. This data changes each frame so lets call it frame data.

Each of these entities have a function that needs to be run each frame lets call it run(). Function run() requires frequently reading (but never writing) to the frame data. Lets also assume that frame data is in the heap and thus not cloned when new threads are made.

These entities may all be run() sequentially on a single thread or if the platform running this code would benefit the entities might be batched into several pthreads or each run() on their own pthread.

So essentially each frame frame data gets updated then every entity gets run() in an arbitrary order on an arbitrary thread.

I realize that reads will take the same amount of time no-matter what but what I am worried about is threads getting blocked while waiting for another thread to finish reading the simulation data. Is this a valid concern at all?

Disregarding the cost of copying would it be a good idea for me to make a copy of the simulation data for each thread I create or will the CPU be fine with multiple threads reading this resource? How would this change if simulation data was on the stack.

Answer 1

You appear to have a classic single writer/multi-reader queue. There are a few strategies for synchronization include RW-lock and Circular Ring Buffers

Having a single copy is definitely the best strategy due to caching. In modern architectures, the queue entries will be read into a cache e.g. L3 by the first thread. Subsequent threads on the same CPU will avoid the re-reading the memory from RAM. Multiple copies of the queue entry may exist in cache across multiple CPUs.

Threads will be time sliced on the CPU so that only one is running at a time. Threads will not block each other in the sense of serial lock like a mutex when reading the same data.

It will be a bad idea to make multiple copies for each thread. The copies will stale the cache even though that data is exactly the same.

Stacks are not shared across threads. If the queue element is on the local stack of a thread then it is a private copy. If the queue element is accessed via pointer to a master thread's stack then only a single copy will be present. Having one thread access the stack of another thread is generally a terrible idea however as race conditions are likely.