Stack Overflow
Stack Overflow

Questions tagged [rayon]

A Rust data-parallelism library that makes it easy to convert sequential computations into parallel Rayon is lightweight and convenient for introducing parallelism into existing code. It guarantees data-race free executions and takes advantage of parallelism when sensible, based on work-load at runtime.

See the official documentation for more information.

123 questions
22
votes
5 answers

Simultaneous mutable access to arbitrary indices of a large vector that are guaranteed to be disjoint

Context I have a case where multiple threads must update objects stored in a shared vector. However, the vector is very large, and the number of elements to update is relatively small. Problem In a minimal example, the set of elements to update can…
Thierry
  • 1,099
  • 9
  • 19
16
votes
2 answers

Per-thread initialization in Rayon

I am trying to optimize my function using Rayon's par_iter(). The single threaded version is something like: fn verify_and_store(store: &mut Store, txs: Vec) { let result = txs.iter().map(|tx| { tx.verify_and_store(store) …
Tomas
  • 5,067
  • 1
  • 35
  • 39
11
votes
2 answers

How do I use Rayon with an existing iterator?

I turn a regex into a HashSet after doing some filtering. I am trying to use it with Rayon, but I can't figure out how to make Rayon work with an existing iterator without converting it to a vector first. Is this possible? let re =…
user964375
  • 2,201
  • 3
  • 26
  • 27
9
votes
2 answers

How can I change the number of threads Rayon uses?

I'm using the Rayon library: extern crate rayon; const N: usize = 1_000_000_000; const W: f64 = 1f64/(N as f64); fn f(x: f64) -> f64 { 4.0/(1.0+x*x) } fn main() { use rayon::prelude::*; let sum : f64 = (0..N) …
asyraaf azhar
  • 383
  • 4
  • 12
8
votes
1 answer

Using the `Result>` trick in Rust, but across multiple threads

It's a common trick to annotate your functions with returning Result> to allow them to return any error at all. However, you can't return this from a thread without the error itself implementing Send. For example this code: use…
Migwell
  • 18,631
  • 21
  • 91
  • 160
8
votes
1 answer

How to use rayon's .par_iter() with a vector of generics?

This is a contrived example but I believe if I can get this working I can apply it to my specific case. extern crate num; extern crate rayon; use rayon::prelude::*; use num::Float; fn sqrts(floats: &Vec) -> Vec { …
davidMcneil
  • 197
  • 3
  • 11
6
votes
1 answer

Is it possible to combine Rayon and Faster?

Rayon looks great for algorithm parallelization of collections, and Faster is great for vectorization (SIMD) on the x86 platform for collections like Vec. I've tried to combine them and the iterators don't seem to like each other. Is there a…
Stan Prokop
  • 5,579
  • 3
  • 25
  • 29
6
votes
2 answers

Replace iter() with par_iter(): cannot borrow data mutably in a captured outer variable in an `Fn` closure

I was hoping to replace an iter() with Rayon's par_iter() in a rather simple case like this, but I am failing to do so. The previous code: indexes_to_increment .iter() .for_each(|x| self.some_data[*x as usize] += 1);` Here's the…
manonthemat
  • 6,101
  • 1
  • 24
  • 49
5
votes
1 answer

Only 1/4th of max memory available when rust wasm compiled with +atomics flag webassembly

So, I've been running out of memory with wasm/rust with +atomic flag and wanted to check how much memory is practically available. Here is my crude minimal working example that logs the memory of a vector before it panics: index.js import init from…
ste_kwr
  • 820
  • 1
  • 5
  • 21
5
votes
2 answers

How to convert ParallelIterator back to sequential Iterator?

I'm iterating over several gigabytes of input items from a database. On each input item, I'm doing some CPU-intensive processing which produces one or more new output items, tens of gigabytes in total. The output items are then stored in another…
Thomas
  • 174,939
  • 50
  • 355
  • 478
5
votes
2 answers

Rust `std::time::Instant` "panicked at 'supplied instant is later than self"

I'm trying to get a simple timer set up in rust that returns true at a certain frequency. #[derive(Clone, Debug)] pub struct IntervalTimer { pub period: Duration, pub delta: Instant, } impl IntervalTimer { pub fn new(period: Duration)…
Michael
  • 507
  • 4
  • 11
5
votes
3 answers

How can I use Rayon to split a big range into chunks of ranges and have each thread find within a chunk?

I am making a program that brute forces a password by parallelization. At the moment the password to crack is already available as plain text, I'm just attempting to brute force it anyway. I have a function called generate_char_array() which, based…
Attila Szeremi
  • 5,235
  • 5
  • 40
  • 64
5
votes
2 answers

Cannot use Rayon's `.par_iter()`

I have a struct which implements Iterator and it works fine as an iterator. It produces values, and using .map(), I download each item from a local HTTP server and save the results. I now want to parallelize this operation, and Rayon looks…
Amandasaurus
  • 58,203
  • 71
  • 188
  • 248
4
votes
2 answers

Efficiently build a Polars DataFrame row by row in Rust

I would like to create a large Polars DataFrame using Rust, building it up row by row using data scraped from web pages. What is an efficient way to do this? It looks like the DataFrame should be created from a Vec of Series rather than adding rows…
Steven Murdoch
  • 151
  • 1
  • 5
4
votes
1 answer

Accessing fields of different index in collection inside par_iter_mut

the following example illustrates what i am trying to do: use rayon::prelude::*; struct Parent { children: Vec, } struct Child { value: f64, index: usize, //will keep the distances to other children in the children vactor of…
Foveng
  • 47
  • 5
1
2 3
…
8 9