How to spawn threads from a vector of structs to run an impl fn from each struct in the vector?

Question

I have a struct which contains a Vec of instances of another base class of struct. I am trying to iterate over the Vec and spawn threads which each run a single impl fn from the base struct. Nothing needs mutable access at any time after the iteration of thread spawning begins; just some basic math returning an f64 (based on values in a HashMap using keys stored in a fixed Vec in each base struct).

I am running into lifetime issues which I don't fully understand and which the compiler error messages (for once) don't help with.

Here is a stripped down version of what I want to implement (with some annotation of the errors encountered):

struct BaseStruct {
    non_copy_field: Vec<&'static str>,  // BaseStruct has vector members (thus can't implement Copy).
}

impl BaseStruct {
    fn target_function(&self, value_dict: &HashMap<&'static str, f64>) -> f64 {
        // Does some calculations, returns the result.
        // Uses self.non_copy_field to get values from value_dict.
        0.0
    }
}

struct StructSet {
    values: HashMap<&'static str, f64>,  // This will be set prior to passing to base_struct.target_function().
    all_structs: Vec<BaseStruct>,        // Vector to be iterated over.
}

impl StructSet {
    fn parallel_calculation(&self) -> f64 {
        let mut result = 0.0;

        let handles: Vec<_> = self.all_structs.iter().map(|base_struct| {
            // Complains about lifetime here   ^^^^^ or    ^ here if I switch to &base_struct
            thread::spawn(move || {
                base_struct.target_function(&self.values)
            })
        }).collect();

        for process in handles.iter() {
            result += process.join().unwrap();
        };
        // Shouldn't all base_structs from self.all_structs.iter() be processed by this point?
        result
    } // Why does it say "...so that reference does not outlive borrowed content" here?
}

I have been trying various combinations of RwLock/Arc/Mutex wrapping the contents of the fields of StructSet to attempt to gain thread-safe, read-only access to each of the elements iterated/passed, but nothing seems to work. I'm looking to keep the codebase light, but I guess I'd consider rayon or similar as I'll need to follow this same process multiple places in the full module.

Can anyone point me in the correct direction?

Answer 1

Rust's lifetime system needs to know that if you borrow something (i.e. have a reference to it), that underlying value will exist for the whole time that you borrow it. For regular function calls, this is easy, but threads cause problems here - a thread you start may outlive the function you start it from. So you can't borrow values into a thread, you have to move values into a thread (which is why you have to write thread::spawn(move || { not just thread::spawn(|| {.

When you call .iter() on a Vec, the values the iterator produces are references to the values in the Vec - they're being borrowed. So you can't use them as-is from another thread. You need to move some owned value into the thread.

There are a few ways you can go about this:

If you don't need the Vec after your processing, you could switch to use .into_iter() rather than .iter(). This will iterate over the owned values in the Vec, rather than borrowing them, which means you can move them into the threads. But because the Vec is giving up ownership of the items, your Vec stops being usable after that.

If you do need your Vec after, and your values are clonable (i.e. they implement the Clone trait), you could call .iter().cloned() instead of .iter() - this will make copies of each of the values, which you can then move into the thread.

If you need the Vec afterwards, and either your values aren't clonable, or you don't want to clone them (maybe because it's expensive, or because it matters to you that both threads are using the exact same object), you can have your Vec store Arc<BaseStruct> instead of just BaseStruct - but that's not quite enough - you'll also need to explicitly clone the values before moving them into the thread (perhaps by using .iter().cloned() like above).

A downside of using Arcs is that no threads will be able to modify the values in the future. If you want to use Arcs, but you ever want o be able to modify the values in the future, you'll need instead of storing an Arc<BaseStruct>, to store an Arc<Mutex<BaseStruct>> (or an Arc<RwLock<BaseStruct>>. The Mutex ensures that only one thread can be modifying (or indeed reading) the value at a time, and the Arc allows for the cloning (so you can move a copy into the other thread).

Answer 2

I found the issue. Specifically it was the use of self.all_structs.iter().map(). Apparently iter() was requiring static lifetimes.

I switched to the following and it's working now:

fn parallel_calculation(&self) -> f64 {
    let mut handles: Vec<_> = Vec::new();

    for base_struct in &self.all_structs {
        let for_solve = self.for_solve.clone();
        let base_struct = base_struct.clone();

        handles.push(thread::spawn(move || {
            let for_solve = for_solve.read().unwrap();
            base_struct.target_function(&for_solve)
        }));
    };

    let mut result = 0.0;
    for process in handles { result += process.join().unwrap(); };
    return result
}

With Arcs and an RwLock in the main struct as follows:

pub struct StructSet {
    all_structs: Vec<Arc<BaseStruct>>,
    for_solve: Arc<RwLock<HashMap<&'static str, f64>>>,
}