8

This is a contrived example but I believe if I can get this working I can apply it to my specific case.

extern crate num;
extern crate rayon;
use rayon::prelude::*;
use num::Float;

fn sqrts<T: Float>(floats: &Vec<T>) -> Vec<T> {
    floats.par_iter().map(|f| f.sqrt()).collect()
}

fn main() {
    let v = vec![1.0, 4.0, 9.0, 16.0, 25.0];
    println!("{:?}", sqrts(&v));
}

This errors at compile time with "the method par_iter exists but the following trait bounds were not satisfied: &std::vec::Vec<T> : rayon::par_iter::IntoParallelIterator". The code works fine if I use iter instead of par_iter or if I switch to using f32 or f64 instead of the generic.

What can I do to be able to use par_iter on a vector of generics? Is the IntoParallelIterator trait meant to be implemented by the end user? How would I go about doing that?

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
davidMcneil
  • 197
  • 3
  • 11

1 Answers1

9

First, read Why is it discouraged to accept a reference to a String (&String) or Vec (&Vec) as a function argument?. Then...

Review the implementors of IntoParallelIterator:

impl<'data, T: Sync + 'data> IntoParallelIterator for &'data [T]

Adding the Sync bound fixes that issue. Rayon works by potentially using multiple threads, but your original T makes no guarantees about if it is safe to share or between threads! This comes up a second time:

error: no method named `collect` found for type `rayon::par_iter::map::Map<rayon::par_iter::slice::SliceIter<'_, T>, rayon::par_iter::map::MapFn<[closure@src/main.rs:7:27: 7:39]>>` in the current scope
 --> src/main.rs:7:41
  |
7 |     floats.par_iter().map(|f| f.sqrt()).collect()
  |                                         ^^^^^^^
  |
  = note: the method `collect` exists but the following trait bounds were not satisfied: `rayon::par_iter::map::MapFn<[closure@src/main.rs:7:27: 7:39]> : rayon::par_iter::map::MapOp<&_>`, `rayon::par_iter::map::Map<rayon::par_iter::slice::SliceIter<'_, T>, rayon::par_iter::map::MapFn<[closure@src/main.rs:7:27: 7:39]>> : std::iter::Iterator`

Checking out collect:

fn collect<C>(self) -> C 
    where C: FromParallelIterator<Self::Item>

We can see that the target type needs to implement FromParallelIterator:

impl<T> FromParallelIterator<T> for Vec<T> where T: Send

Thus, adding both bounds allows it to compile:

fn sqrts<T: Float + Send + Sync>(floats: &[T]) -> Vec<T>
Community
  • 1
  • 1
Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
  • I guess it follows the std lib, whereby `collect` delegates to a `FromXXX` iterator implementation... but, there is _nothing_ here that would point me to looking at `FromParallelIterator` for the hint that helps fix the problem. Well, not in a straight forward way anyway. – Simon Whitehead Jan 10 '17 at 19:50
  • @SimonWhitehead hmm. I did neglect to add that part to my answer, but it's still a bit hollow. The error message indicates `Map` as not implementing `collect`, but really it's not really `Map` that is at fault... – Shepmaster Jan 10 '17 at 19:53
  • Ah, I can see it now with your updates. If I were not so obsessed with Rust I would certainly be very confused about the error here though and probably give up. Sometimes its very clear to see why so many people drop Rust after a short time. Thanks for the edit to make it a bit clearer! – Simon Whitehead Jan 10 '17 at 21:36
  • 2
    `Send` and `Sync` will both show up a lot when it comes to threading! The logic here should be simple. `par_iter` means distributing `&T` to different threads. That means `T: Sync`. Map is returning `T`s by value (and passing them to potential new threads) that requires `T: Send`. `collect` requires `T: Send` the same way. Rayon can possibly help by tightening trait bounds up front: https://github.com/nikomatsakis/rayon/issues/204 – bluss Jan 10 '17 at 21:43
  • @bluss Only with my (almost) 12 months of Rust behind me can I grok that requirement by looking at the Rust documentation, rayons function/type signatures and their trait bounds. I'm just recalling back to 9 months ago when the above error message would have completely confused me :) A nice idea to have rayon try to help in this regard - I will follow the issue! – Simon Whitehead Jan 10 '17 at 22:25
  • I'm suggesting the Send/Sync bounds should also be understandable from principles without looking at the API – bluss Jan 11 '17 at 09:16
  • @SimonWhitehead i am a Rust dropper partly for what you mentioned (partly because API moved too fast, this was a year or two ago)... but im thinking of giving Rust another chance because more and more examples i see are relatively clean looking. I can recall being dumbfounded by bash scripts and 200 page long C++ template errors and 10,000 line build script fails in the past... somehow I got used to them. And they are ridiculously complicated. Maybe any system that has people helping each other figure stuff out on StackOverflow is hopefully heading in a good direction. – don bright Sep 12 '17 at 04:15