How can I read and modify a variable using Rayon's par_iter in a thread-safe manner?

Question

This code:

use rayon::prelude::*; // 1.5.0

fn main() {
    let mut items = Vec::new();
    items.push("hello");
    items.push("foo");
    items.push("bar");
    items.push("ipsum");

    let mut counter = 0;

    let results = items.par_iter().map(|item| {
        // do something time consuming with item
        counter += 1;
        print!("completed {} items\r", counter);
        0
    });
}

Produces an error:

warning: unused variable: `item`
  --> src/main.rs:12:41
   |
12 |     let results = items.par_iter().map(|item| {
   |                                         ^^^^ help: if this is intentional, prefix it with an underscore: `_item`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: unused variable: `results`
  --> src/main.rs:12:9
   |
12 |     let results = items.par_iter().map(|item| {
   |         ^^^^^^^ help: if this is intentional, prefix it with an underscore: `_results`

error[E0594]: cannot assign to `counter`, as it is a captured variable in a `Fn` closure
  --> src/main.rs:14:9
   |
14 |         counter += 1;
   |         ^^^^^^^^^^^^ cannot assign

Answer 1

Rust prevents you here from having a data race by writing to the same variable from two different threads. You have a couple of options how to solve this. It really depends on the specific circumstances.

1. The simplest is to use a Mutex for counter. This allows you safe access to the same variable. Introducing the Mutex has the risk of eating up all the speedup of the parallel iterator, since everything will get sequential through the Mutex access. This can be acceptable if the runtime of the map is large and locking the Mutex short.
2. For the specific case of counters atomic types such as AtomicI32 work well, but they are hard or impossible to use for more complex types.
3. Instead of directly aggregating on a single variable, the work can be done multiple times in parallel and then merged together. This is what the reduce-functions from rayon do. Each thread will have at least one counter and they will be merged together to produce a single final result.