Max Degree of Parallelism for AsParallel()

Question

While using Parallel.ForEach we have the option to define the Parallel options and set the Max Degree of Parallelism like :

Parallel.ForEach(values, new ParallelOptions {MaxDegreeOfParallelism = number}, value = > {
    // Do Work
})

But while doing PLINQ like:

Tabel.AsEnumberable()
     .AsParallel()
     .Where(//Logic)

I was not able to find a way to set MaxDegreeOfParallelism. I looked up on the net as well, but didn't find anything. As anyone found a way around this? Any help is appreciated.

Yuval Itzchakov · Accepted Answer · 2015-06-13T08:06:29.373

67

You can use ParallelEnumerable.WithDegreeOfParallelism:

Sets the degree of parallelism to use in a query. Degree of parallelism is the maximum number of concurrently executing tasks that will be used to process the query.

var result = Tabel.AsEnumberable()
                  .AsParallel()
                  .WithDegreeOfParallelism(number)
                  .Where(/* predicate */);

Edit:

@svick provided an excellent on ParallelOptions.MaxDegreeOfParallelism vs PLINQ’s WithDegreeOfParallelism which emphasizes the difference between the two:

Parallel works using an under-the-covers concept we refer to as replicating tasks. The concept is that a loop will start with one task for processing the loop, but if more threads become available to assist in the processing, additional tasks will be created to run on those threads. This enables minimization of resource consumption. Given this, it would be inaccurate to state that ParallelOptions enables the specification of a DegreeOfParallelism, because it’s really a maximum degree: the loop starts with a degree of 1, and may work its way up to any maximum that’s specified as resources become available.

PLINQ is different. Some important Standard Query Operators in PLINQ require communication between the threads involved in the processing of the query, including some that rely on a Barrier to enable threads to operate in lock-step. The PLINQ design requires that a specific number of threads be actively involved for the query to make any progress. Thus when you specify a DegreeOfParallelism for PLINQ, you’re specifying the actual number of threads that will be involved, rather than just a maximum.

edited Jun 13 '15 at 08:06

answered Aug 13 '14 at 15:41

Yuval Itzchakov

146,575
32
257
321

4

You might also want to read [*ParallelOptions.MaxDegreeOfParallelism vs PLINQ’s WithDegreeOfParallelism*](http://blogs.msdn.com/b/pfxteam/archive/2009/05/29/9655514.aspx) on the difference between the two. – svick Aug 13 '14 at 20:49
This answer appears to be at odds with [this one](http://stackoverflow.com/a/1812670/538763) – crokusek May 12 '15 at 18:52
@crokusek I'm not sure where that answer got its references from, but it doesn't seem to back itself up by any offical docs. Also, I'm not sure where they got the notion that the first IO operations would block all other operations. Sounds peculiar. – Yuval Itzchakov May 12 '15 at 19:25
1

Unfortunately I am seeing behavior in line with the other answer--that is that specifying WithDegreeOfParallelism(32) is still hard limited to 8 on a 4 core machine. Does anyone have a proof where the value was used directly where N threads >> 2*core? – crokusek May 13 '15 at 00:21
From a small test I made. it doesn’t seem that the actual number of threads equals DegreeOfParallelism . I assigned DegreeOfParallelism = 100 , and ran a small program that records for each “task” it’s thread id – only 9 threads were actually used. I was running in .Net 4.5.1 – omer schleifer Sep 07 '16 at 04:36
@omer Did you test have enough work for 100 threads to run concurrently? – Yuval Itzchakov Sep 07 '16 at 07:15
Each work item consisted of thread.sleep, plus writes thread id. Only 9 threads were used each ran multiple work items – omer schleifer Sep 07 '16 at 19:34
@omerschleifer Post a gist of your test, I'll take a look at it. – Yuval Itzchakov Sep 08 '16 at 06:12

score 9 · Answer 2 · answered Aug 13 '14 at 15:42

9

Yes, you can certainly do that. You just use WithDegreeOfParallelism extension method

yourSequence.AsParallel()
    .WithDegreeOfParallelism(5)//Whatever number as you like
    .Where(...);

answered Aug 13 '14 at 15:42

Sriram Sakthivel

72,067
7
111
189

score 0 · Answer 3 · answered Jun 12 '15 at 18:53

0

<IEnumerable>.AsParallel()
.WithDegreeOfParallelism(n)
.Where(x=>)

answered Jun 12 '15 at 18:53

letscode

33
3

1

@codingpirate Just wondering, you unmarked my answer as accepted for an answer with an identical answer, almost a year later? Any particular reason why? – Yuval Itzchakov Jun 13 '15 at 07:31
@YuvalItzchakov - To close the thread I accepted the ans. Having said that I also have your explanation as an accepted answer as well – codingpirate Jun 13 '15 at 19:05

Max Degree of Parallelism for AsParallel()

3 Answers3

Linked