Parallel map operations?

Question

Does Scala provide a way to execute parallel map operations as part of the standard language?

For example, given:

scala> val a = List((1,2), (3,4), (3,6))
a: List[(Int, Int)] = List((1,2), (3,4), (3,6))

I can do:

scala> a.map(tup => tup._1 + tup._2)
res0: List[Int] = List(3, 7, 9)

However, to the best of my knowledge this maps the provided function over the list objects sequentially. Is there a built-in way to have the function applied to each element in a separate thread (or equivalent), and the results then gathered into a resulting list?

Don't use `List` for distributed (`par`) operations. You should use an `IndexedSeq`. — senia, Nov 20 '13 at 10:34
@senia - can you just elaborate shortly on why using a list is a bad idea in this case? I do not need the resulting list to be ordered in any way. — csvan, Nov 20 '13 at 11:18
It's because of the way `par` method creates `parallel collection`. For `Vector` (default `IndexedSeq` implementation), `Range` and `Array` it just wraps initial collection with lightweight wrapper. But for `List` it should create a completely new collection, it could lead to performance issue. See [Creating a Parallel Collection](http://docs.scala-lang.org/overviews/parallel-collections/overview.html#creating_a_parallel_collection). — senia, Nov 20 '13 at 11:39
For the sake of precision in language, "parallel" and "distributed" really should not be conflated. — Randall Schulz, Nov 20 '13 at 17:27
Right, fixing that. Also, thanks a lot for the clarification @senia — csvan, Nov 20 '13 at 23:12

score 39 · Accepted Answer · edited Mar 13 '18 at 16:18

39

If you add par then you will get a parallel collection and operations on it will be processed in parallel. To convert back to a normal collection call a toList.

So your code would look like:

a.par.map(tup => tup._1 + tup._2).toList

Or a .seq to get a Sequential Collection (the opposite of a Parallel Collection).

a.par.map(tup => tup._1 + tup._2).seq

Also, check the documentation.

edited Mar 13 '18 at 16:18

belka

1,480
1
18
31

answered Nov 20 '13 at 10:10

Akos Krivachy

4,915
21
28

mikołak · Answer 2 · 2013-11-20T10:14:14.973

8

par splits your list for processing over several threads. You can then regulate how the threading is done by modyfying the tasksupport member of the resultant ParSeq.

edited Nov 20 '13 at 10:14

answered Nov 20 '13 at 10:08

mikołak

9,605
1
48
70

Parallel map operations?

2 Answers2

Linked