1

For the sake of completeness, I've searched and read other articles here, such as:

Parallel.ForEach not spinning up new threads

but they don't seem to address my case, so off we go:

I have a Parallel.ForEach of an array structure, like so:

Dim opts As New ParallelOptions
opts.MaxDegreeOfParallelism = intThreads

Parallel.ForEach(udtExecPlan, opts,
     Sub(udtStep)
         Dim strItem As String = udtStep.strItem

Basically, for each item, I do some nested loops and end up calling a function with those loop assignments as parameters.

The function executes a series of intense calculations (which takes up most of the function's processing time) and records the results on an MSSQL table, and if some conditions are met, the function returns True, else False. If the result is True, then I simply Return from the parallel function Sub(udtStep) and another item from the array should continue. If the result is False, I simply go through another interation of the deepest nested loop, and so on, working towards the completion of the other outer loops, etc. So, in a nutshell, all nested loops are inside the main Parallel.ForEach loop, like so:

Parallel.ForEach(udtExecPlan, opts,
    Sub(udtStep)
        Dim strItem As String = udtStep.strItem

        If Backend.Exists(strItem) Then Return                                                       

        For intA As Integer = 1 To 5
            For intB As Integer = 1 To 50
                Dim booResult As Boolean = DoCalcAndDBStuff(strItem, intA, intB)
                If booResult = True Then Return
            Next intB
        Next intA
    End Sub)

It is important to notice that udtExecPlan has about 585 elements. Each item takes from 1 minute to several hours to complete.

Now, here's the problem:

Whether I do this:

Dim opts As New ParallelOptions
opts.MaxDegreeOfParallelism = intThreads

Parallel.ForEach(udtExecPlan, opts,

where intThreads is the core count, or whatever number I assign to it (tried 5, 8, 62, 600), or whether I simply omit the whole the ParallelOptions declaration and opts from the Parallel.ForEach statement, I notice it will spin up as many threads I have specified upto the total amount of cores (including HT cores) in my system. That is all fine and well.

For example, on an HP DL580G7 server with 32 cores / 64 HT cores and 128GB RAM, I can see 5, 8, 62 or 64 (using the 600 option) threads busy on the Task Manager, which is what I'd expect.

However, as the items on the array are processed, the threads on Task Manager "die off" (go from around 75% utilization to 0%) and are never spun up again, until only 1 thread is working. For example, if I set intThreads to 62 (or unlimited if I omitted the whole ParallelOptions declaration and opts from the Parallel.ForEach statement), I can see on the db table that 62 (or 64) items have been processed, but from then on, it just falls back to 1 thread and 1 item at a time.

I was expecting that a new thread would be spun up as soon as an item was done, as there are some 585 items to go through. It is almost as if 62 or 64 items are done in parallel and then on only 1 item until completion, which renders the whole server practically idling thereafter.

What am I missing?

I have tried some other different processes with a main Parallel.For loop (no other outer loop present, just as in this example) and get the same behaviour.

Why? Any thoughts welcome.

I am using VS.NET 2015 with .NET Framework 4.6.1, W2K8R2 fully patched.

Thanks!

Lance U. Matthews
  • 15,725
  • 6
  • 48
  • 68
Molasar
  • 184
  • 1
  • 8
  • What type of application is it? For example any kind of application running in IIS can always look strange, because it does a lot of "magic" behind the scenes of thread management. – Al Kepp Jul 22 '18 at 18:46
  • I'd be impressed that server can do much of anything at all with only 128 MB of RAM. (Or it's just a typo...) – Lance U. Matthews Jul 22 '18 at 18:50
  • It is just a regular console app. It outputs some sporadic messages on the console, but other than that, all else is pretty straightforward. – Molasar Jul 22 '18 at 18:52
  • Sorry. Typo. 128GB RAM. Fixed. – Molasar Jul 22 '18 at 18:54
  • 1
    Is `udtExecPlan` mostly minutes-long or hours-long tasks? Does that collection tend to be ordered by computational complexity? Perhaps 63 of your cores are being assigned a batch of tasks that complete quickly and the 64th is getting stuck with all the long tasks (although that would mean 63 cores are getting ~63 tasks and #64 is getting ~521 tasks, instead of the 9 tasks/core one might expect.) Still, have you tried an overload of `ForEach` that takes a [`Partitioner`](https://docs.microsoft.com/dotnet/api/system.collections.concurrent.partitioner) with dynamic partitioning enabled? – Lance U. Matthews Jul 22 '18 at 19:16
  • No, it is not ordered by computational complexity. I would say the average is minutes long... about 3 to 4 minutes on average. About the 64th getting all the hard work, it might be, but why would the framework not notice this and span all the work across all the available threads? I will try Partitioner and post back. – Molasar Jul 22 '18 at 19:24
  • However, being that the computations are on average over 3 minutes long, would this provide any benefit? – Molasar Jul 22 '18 at 19:30
  • In the past, and even now, I have solved this running the exe several times spaced 10 minutes apart. Since the If Backend.Exists(strItem) Then Return statement effectively skips all processed or processing items, this balances the whole workload. However, this seems far from an ideal solution. – Molasar Jul 22 '18 at 19:33
  • With static partitioning, the default, which tasks get runs on which cores is determined up-front, so it's not monitoring task execution for "fairness"/balance and, even if it did, it'd be too late to change anything. If you want it to spread out the tasks evenly - so each core always takes the next pending task - you need to explicitly provide a `Partitioner` that will do so. See [Custom Partitioners for PLINQ and TPL](https://learn.microsoft.com/dotnet/standard/parallel-programming/custom-partitioners-for-plinq-and-tpl). – Lance U. Matthews Jul 22 '18 at 20:00
  • In this line `If booResult = True Then Return` the `Return` statement will exit the entire function that the code is in. Is that what you want, or do you just wish to exit one or both of the inner loops? – David Wilson Jul 23 '18 at 05:28
  • Yes, it should exit the entire function since a solution has been found for the current item (in the array). – Molasar Jul 24 '18 at 13:28

0 Answers0