How can I send a function to another thread?

Question

I am attempting to write a simpler unit test runner for my Rust project. I have created a TestFixture trait that my test fixture structs will implement, similar to inheriting from the unit test base class in other testing frameworks. The trait is fairly simple. This is my test fixture

pub trait TestFixture {
    fn setup(&mut self) -> () {}
    fn teardown(&mut self) -> () {}
    fn before_each(&mut self) -> () {}
    fn after_each(&mut self) -> () {}
    fn tests(&mut self) -> Vec<Box<Fn(&mut Self)>>
        where Self: Sized {
        Vec::new()
    }
}

My test running function is as follows

pub fn test_fixture_runner<T: TestFixture>(fixture: &mut T) {
    fixture.setup();

    let _r = fixture.tests().iter().map(|t| {
        let handle = thread::spawn(move || {
            fixture.before_each();
            t(fixture);
            fixture.after_each();
        });

        if let Err(_) = handle.join() {
            println!("Test failed!")
        } 
    });

    fixture.teardown();
}

I get the error

src/tests.rs:73:22: 73:35 error: the trait `core::marker::Send` is not implemented for the type `T` [E0277]
src/tests.rs:73         let handle = thread::spawn(move || {
                                     ^~~~~~~~~~~~~
note: in expansion of closure expansion
src/tests.rs:69:41: 84:6 note: expansion site
src/tests.rs:73:22: 73:35 note: `T` cannot be sent between threads safely
src/tests.rs:73         let handle = thread::spawn(move || {
                                     ^~~~~~~~~~~~~
note: in expansion of closure expansion
src/tests.rs:69:41: 84:6 note: expansion site
src/tests.rs:73:22: 73:35 error: the trait `core::marker::Sync` is not implemented for the type `for<'r> core::ops::Fn(&'r mut T)` [E0277]
src/tests.rs:73         let handle = thread::spawn(move || {
                                     ^~~~~~~~~~~~~
note: in expansion of closure expansion
src/tests.rs:69:41: 84:6 note: expansion site
src/tests.rs:73:22: 73:35 note: `for<'r> core::ops::Fn(&'r mut T)` cannot be shared between threads safely
src/tests.rs:73         let handle = thread::spawn(move || {
                                     ^~~~~~~~~~~~~
note: in expansion of closure expansion

I have tried adding Arcs around the types being sent to the thread, no dice, same error.

pub fn test_fixture_runner<T: TestFixture>(fixture: &mut T) {
    fixture.setup();

    let fix_arc = Arc::new(Mutex::new(fixture));
    let _r = fixture.tests().iter().map(|t| {
        let test_arc = Arc::new(Mutex::new(t));
        let fix_arc_clone = fix_arc.clone();
        let test_arc_clone = test_arc.clone();
        let handle = thread::spawn(move || {
            let thread_test = test_arc_clone.lock().unwrap();
            let thread_fix = fix_arc_clone.lock().unwrap();
            (*thread_fix).before_each();
            (*thread_test)(*thread_fix);
            (*thread_fix).after_each();
        });

        if let Err(_) = handle.join() {
            println!("Test failed!")
        } 
    });

    fixture.teardown();
}

A sample test fixture would be something like

struct BuiltinTests {
    pwd: PathBuf
}

impl TestFixture for BuiltinTests {
    fn setup(&mut self) {
        let mut pwd = env::temp_dir();
        pwd.push("pwd");

        fs::create_dir(&pwd);
        self.pwd = pwd;
    }

    fn teardown(&mut self) {
        fs::remove_dir(&self.pwd);
    }

    fn tests(&mut self) -> Vec<Box<Fn(&mut BuiltinTests)>> {
        vec![Box::new(BuiltinTests::cd_with_no_args)]
    }
}

impl BuiltinTests {
    fn new() -> BuiltinTests {
        BuiltinTests {
            pwd: PathBuf::new()
        }
    }
}

fn cd_with_no_args(&mut self) {
    let home = String::from("/");
    env::set_var("HOME", &home);

    let mut cd = Cd::new();
    cd.run(&[]);

    assert_eq!(env::var("PWD"), Ok(home));
}

#[test]
fn cd_tests() {
    let mut builtin_tests = BuiltinTests::new();
    test_fixture_runner(&mut builtin_tests);
}

My whole intention of using threads is isolation from the test runner. If a test fails an assertion it causes a panic which kills the runner. Thanks for any insight, I'm willing to change my design if that will fix the panic problem.

Answer 1

There are several problems with your code, I'll show you how to fix them one by one.

The first problem is that you're using map() to iterate over an iterator. It won't work correctly because map() is lazy - unless you consume the iterator, the closure you passed to it won't run. The correct way is to use for loop:

for t in fixture().tests().iter() {

Second, you're iterating the vector of closures by reference:

fixture.tests().iter().map(|t| {

iter() on a Vec<T> returns an iterator yielding items of type &T, so your t will be of type &Box<Fn(&mut Self)>. However, Box<Fn(&mut T)> does not implement Sync by default (it is a trait object which have no information about the underlying type except that you specified explicitly), so &Box<Fn(&mut T)> can't be used across multiple threads. That's what the second error you see is about.

Most likely you don't want to use these closures by reference; you probably want to move them to the spawned thread entirely. For this you need to use into_iter() instead of iter():

for t in fixture.tests().into_iter() {

Now t will be of type Box<Fn(&mut T)>. However, it still can't be sent across threads. Again, it is a trait object, and the compiler does not know if the type contained inside is Send. For this you need to add Send bound to the type of the closure:

fn tests(&mut self) -> Vec<Box<Fn(&mut Self)+Send>>

Now the error about Fn is gone.

The last error is about Send not being implemented for T. We need to add a Send bound on T:

pub fn test_fixture_runner<T: TestFixture+Send>(fixture: &mut T) {

And now the error becomes more comprehensible:

test.rs:18:22: 18:35 error: captured variable `fixture` does not outlive the enclosing closure
test.rs:18         let handle = thread::spawn(move || {
                                ^~~~~~~~~~~~~
note: in expansion of closure expansion
test.rs:18:5: 28:6 note: expansion site
test.rs:15:66: 31:2 note: captured variable is valid for the anonymous lifetime #1 defined on the block at 15:65
test.rs:15 pub fn test_fixture_runner<T: TestFixture+Send>(fixture: &mut T) {
test.rs:16     fixture.setup();
test.rs:17
test.rs:18     for t in fixture.tests().into_iter() {
test.rs:19         let handle = thread::spawn(move || {
test.rs:20             fixture.before_each();
           ...
note: closure is valid for the static lifetime

This error happens because you're trying to use a reference in a spawn()ed thread. spawn() requires its closure argument to have 'static bound, that is, its captured environment must not contain references with non-'static lifetimes. But that's exactly what happens here - &mut T is not 'static. spawn() design does not prohibit avoiding joining, so it is explicitly written to disallow passing non-'static references to the spawned thread.

Note that while you're using &mut T, this error is unavoidable, even if you put &mut T in Arc, because then the lifetime of &mut T would be "stored" in Arc and so Arc<Mutex<&mut T>> also won't be 'static.

There are two ways to do what you want.

First, you can use the unstable thread::scoped() API. It is unstable because it is shown to allow memory unsafety in safe code, and the plan is to provide some kind of replacement for it in the future. However, you can use it in nightly Rust (it won't cause memory unsafety by itself, only in specifically crafted situations):

pub fn test_fixture_runner<T: TestFixture+Send>(fixture: &mut T) {
    fixture.setup();

    let tests = fixture.lock().unwrap().tests();
    for t in tests.into_iter() {
        let f = &mut *fixture;

        let handle = thread::scoped(move || {
            f.before_each();
            t(f);
            f.after_each();
        });

        handle.join();
    }

    fixture.teardown();
}

This code compiles because scoped() is written in such a way that it guarantees (in most cases) that the thread won't outlive all captured references. I had to reborrow fixture because otherwise (because &mut references aren't copyable) it would be moved into the thread and fixture.teardown() would be prohibited. Also I had to extract tests variable because otherwise the mutex will be locked by the main thread for the duration of the for loop which would naturally disallow locking it in the child threads.

However, with scoped() you can't isolate the panic in the child thread. If the child thread panics, this panic will be rethrown from join() call. This may or may not be a problem in general, but I think it is a problem for your code.

Another way is to refactor your code to hold the fixture in Arc<Mutex<..>> from the beginning:

pub fn test_fixture_runner<T: TestFixture + Send + 'static>(fixture: Arc<Mutex<T>>) {
    fixture.lock().unwrap().setup();

    for t in fixture.lock().unwrap().tests().into_iter() {
        let fixture = fixture.clone();

        let handle = thread::spawn(move || {
            let mut fixture = fixture.lock().unwrap();

            fixture.before_each();
            t(&mut *fixture);
            fixture.after_each();
        });

        if let Err(_) = handle.join() {
            println!("Test failed!")
        } 
    }

    fixture.lock().unwrap().teardown();
}

Note that now T has also to be 'static, again, because otherwise it couldn't be used with thread::spawn() as it requires 'static. fixture inside the inner closure is not &mut T but a MutexGuard<T>, and so it has to be explicitly converted to &mut T in order to pass it to t.

This may seem overly and unnecessarily complex, however, such design of a programming language does prevent you from making many errors in multithreaded programming. Each of the above errors we have seen is valid - each of them would be a potential cause of memory unsafety or data races if it was ignored.

Answer 2

As stated in the Rust HandBook's Concurrency section:

When a type T implements Send, it indicates to the compiler that something of this type is able to have ownership transferred safely between threads.

If you do not implement Send, ownership cannot be transfered between threads.