1

I have a number of questions regarding the operation of a Parallel.ForEach loop, especially in regards to the setting of theParallelOptions.MaxDegreeOfParallelism property.

My computer's CPU is Quad core featuring 8 logical processors.

To me the following should be the maximum number of possible process that can be performed in work in parallel as after all, I have 8 threads available:

            ParallelOptions parallelOptions = new()
            {
                MaxDegreeOfParallelism = Environment.ProcessorCount //8,
            };

Consider the following code. This simply iterates around a list of 300 uri's and does "something" with the responses:

            List<string> uriList = new List<string>();

            for (int i = 0; i < 300; i++)
            {
                uriList.Add(uri);
            }

            HttpClient httpClient = new HttpClient();

            ParallelOptions parallelOptions = new()
            {
                MaxDegreeOfParallelism = uriList.Count(),
            };

            await Parallel.ForEachAsync(uriList, parallelOptions, async (uri, token) =>
            {
                var response = await httpClient.GetStringAsync(uri);

                if (response != null)
                {
                    ProcessResponse(response);
                }
            });

Note that the MaxDegreeOfParallelism property is set to the size of the uriList i.e 300 rather than the thread count available to me from my physical hardware of 8. This code works and I'm lost to why setting the MaxDegreeOfParallelism property that high "works".

Questions:

  1. The MaxDegreeOfParallelism property can be specified to any number but the maximum amount of concurrent operations will only ever be as high as the hardware's available thread count?

  2. The MaxDegreeOfParallelism property can be though of as setting how many parallel "batches" of work will be carried out concurrently? For example iterating round a list of 16 items with a MaxDegreeOfParallelism set to '8' will causes two batches of '8' concurrent calls?

  3. If the MaxDegreeOfParallelism property isn't set then by default the maximum number of threads avaliable will be set?

  4. In the situation of an asynchronous Parallel.Foreach do procedding requests wait until the previous "batched" calls have returned? Or, when an await is encountered and a thread is "freed" can another item in the uriList begin its logical steps in the loop?

  5. Is there a "sweet spot" for the setting of the MaxDegreeOfParallelism property?

LostInParallel
  • 23
  • 3
  • Do not conflate number of processor cores with number of threads. My chosen sweet spot is to set `MaxDegreeOfParallelism` to `2 * Environment.ProcessorCount`. – Rick Davin Oct 25 '22 at 20:07
  • The only limit to your thread count is your imagination (and system memory). – AndyG Oct 25 '22 at 20:36

1 Answers1

0

Your code works as desired, because I/O-bound asynchronous operations for the most part don't use threads.

The Parallel family of methods schedule work on the ThreadPool by default. You can customize where the work is scheduled by providing a custom TaskScheduler, but this is rarely necessary. In most cases using the ThreadPool is OK. The ThreadPool can create a number of threads much larger than the number of the processors/cores. If need to, it can create thousands of threads¹. All these threads will share the available processors/cores of the machine. If there are more active (non sleeping) threads than cores, the operating system will split the available processing power to all threads, so all threads will get their time slices and make progress.

The MaxDegreeOfParallelism doesn't work with batches. When the processing of an element completes, immediately starts the processing of another element. It doesn't wait for the previous batch to complete, before starting the next batch.

The default MaxDegreeOfParallelism for the Parallel.ForEachAsync is equal to the Environment.ProcessorCount, and for all other Parallel methods it is equal to -1, which means unlimited parallelism. My suggestion is to specify always the MaxDegreeOfParallelism when using any Parallel method, which is different from what Microsoft recommends.

The sweet spot for the MaxDegreeOfParallelism for CPU-bound operations is Environment.ProcessorCount, although oversubscription might help if the workload is unbalanced. The Parallel.ForEachAsync is used typically for I/O-bound asynchronous operations, where the sweet spot depends on the capabilities of the remote server, or the bandwidth of the network.

¹ For more details about the behavior of the ThreadPool, you can look here.

Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
  • So in theory is the hardware the only limiting factor on setting the MaxDegreeOfParallelism property? If an 8 core CPU is capable of using the Thread pool to spawn thousands of handlers for (in my example) a list of 300 items, would a 16 core CPU be able to spawn even more and get the job done faster? – LostInParallel Oct 25 '22 at 22:25
  • @LostInParallel in your example you are calling the `HttpClient.GetStringAsync` method. This method is genuinely asynchronous. It's not using threads while it is in-flight. You can have hundreds of concurrent `HttpClient.GetStringAsync` operations, using only a handful of threads. Regardless of the number of cores, the whole operation will complete at roughly the same time. A single-core machine should be able to handle it just as well as an 64-core machine. – Theodor Zoulias Oct 25 '22 at 22:45
  • Thanks again for a quick and concise reply. So in the scenario of a non async situation where a thread isn't released, for example when used in an operation where a `Parallel.ForEach` is appropriate, will the threads still be taken from the `ThreadPool`? How does the blocking nature of a non-async call affect this? – LostInParallel Oct 26 '22 at 06:11
  • @LostInParallel yes, the `Parallel.ForEach` schedules all the work on the `ThreadPool`. If the `MaxDegreeOfParallelism` is larger than the number of threads that the `ThreadPool` creates immediately on demand, the `ThreadPool` will become saturated. You can prevent this from happening with the `ThreadPool.SetMinThreads` method. The links inside my answer explain all that. In case the synchronous work is I/O-bound, the number of cores will still be largely irrelevant. The number of cores plays a role only when the work is CPU-bound. – Theodor Zoulias Oct 26 '22 at 09:46