TPL force higher parallelism

Question

When queuing Tasks to the ThreadPool, the code relies on the default TaskScheduler to execute them. In my code example, I can see that 7 Tasks maximum get executed in parallel on separate threads.

new Thread(() =>
{
    while (true)
    {
        ThreadPool.GetAvailableThreads(out var wt, out var cpt);
        Console.WriteLine($"WT:{wt} CPT:{cpt}");
        Thread.Sleep(500);
    }
}).Start();

var stopwatch = new Stopwatch();
stopwatch.Start();
var tasks = Enumerable.Range(0, 100).Select(async i => { await Task.Yield(); Thread.Sleep(10000); }).ToArray();
Task.WaitAll(tasks);
Console.WriteLine(stopwatch.Elapsed.TotalSeconds);
Console.ReadKey();

Is there a way to force the scheduler to fire up more Tasks on other threads? Or is there a more "generous" scheduler in the framework without implementing a custom one?

EDIT:

Adding ThreadPool.SetMinThreads(100, X) seems to do the trick, I presume awaiting frees up the thread so the pool think it can fire up another one and then it immediately resumes.

By default, the minimum number of threads is set to the number of processors on a system. You can use the SetMinThreads method to increase the minimum number ofthreads. However, unnecessarily increasing these values can cause performance problems. If too many tasks start at the same time, all of them might appear to be slow. In most cases, the thread pool will perform better with its own algorithm for allocating threads. Reducing the minimum to less than the number of processors can also hurt performance.

From here: https://msdn.microsoft.com/en-us/library/system.threading.threadpool.setminthreads(v=vs.110).aspx

I removed AsParallel as it is not relevant and it just seems to confuse readers.

@rudolf_franek 16 logical cores. And I think it should be able to interleave more threads at hardware level anyway, shouldn't it? — user4388177, May 24 '17 at 12:44
What is this code trying to do ? It mixes up raw threads *and* tasks *and PLINQ *and* a tiny range, *and* blocking of the processing threads. Any assumed blocking issues are caused by the code itself, not any limitations of PLINQ, Tasks or the thread pool — Panagiotis Kanavos, May 24 '17 at 13:20
@user4388177 `it should be able to interleave more threads at hardware level` no, that is called thrashing and is wasteful. Why **run** more threads at the same time than there are *cores* ? You can't anyway, you'll have to *switch* from one thread to another. PLINQ uses roughly as many **tasks** as there are cores to partition and process a lot of data. Your code though has no data and blocks the worker tasks. By blocking using `Thread.Sleep` inside Select, you've blocked PLINQ — Panagiotis Kanavos, May 24 '17 at 13:22
Second problem. Your PLINQ query generates blocked tasks, so `WaitAll` blocks waiting for 100 sleep operations to finish. Those sleep operations will be generated in batches of 16. That's another way that this code generates delays instead of doing any work. I won't even try to guess whether this results in 6 consecutive sleeps, or more — Panagiotis Kanavos, May 24 '17 at 13:28
@user4388177 if you want to actually test PLINQ, create a simple test, eg generate 1M rows and calculate their squares with `Enumerable.Range(0,1000000).AsParallel().Select(int i => Math.Pow(i)).ToList()`. This will partition the 1M rows to roughly as many partitions as there are cores and process each partition at full speed. — Panagiotis Kanavos, May 24 '17 at 13:29
@user4388177 to put it another way, what you did is equivalent to testing an 8-lane highway by placing 100 rigs across the lanes, including the emergency lane — Panagiotis Kanavos, May 24 '17 at 13:32
I have updated the question to fix the blocking enumerable limited by the degree of parallelism of PLINQ, the fact that I'm using different paradigms is irrelevant. This code is meant just to test how many `Tasks` the default scheduler spawns in parallel. — user4388177, May 24 '17 at 13:46
@MartinLiversage in that way the whole experiment won't make sense anymore. Threads will be free to get back to the pool. — user4388177, May 24 '17 at 13:50
@user4388177 the experiment doesn't make sense anyway. As for threads going back to the pool - that's the idea isnt' it? *Unless* you deal with data parallelism, you want the threads to go back fast. I used the 8-lane highway example for a reason — Panagiotis Kanavos, May 24 '17 at 13:59
@user4388177 are you trying to see whether the task scheduler will increase the number of active tasks if it sees a bottleneck perhaps? — Panagiotis Kanavos, May 24 '17 at 14:00
@PanagiotisKanavos I'm trying to tell the CLR to take more threads from the threadpool and run more tasks in parallel on them regardless of hardware/OS limitations. — user4388177, May 24 '17 at 14:01
@user4388177 And it's not doing it, because it knows that it's a bad idea, and it's right. You shouldn't be doing that. — Servy, May 24 '17 at 14:03
@user4388177 when MVPs and the author of *the* [.NET concurrency book](https://stephencleary.com/book/) tell you that you misunderstand something you shouldn't assume they are wrong, ignorant or too dense to understand your question. What if Jon Skeet told you the same things everyone else does? — Panagiotis Kanavos, May 24 '17 at 14:16
@PanagiotisKanavos I do not dogmatic believe everything a person says just because he is a universally recognized knowledgeable person. That is called religion, this is software development. I treat everyone's opinion in the same way, verifying it is right before accepting it. — user4388177, May 24 '17 at 14:21

score 3 · Answer 1 · answered May 24 '17 at 13:51

3

Is there a way to force the scheduler to fire up more Tasks on other threads?

You cannot have more executing threads than you have CPU cores. This is just how computers work. If you use more threads, then your work will actually get done more slowly since the threads must swap in and out of the cores in order to run.

Or is there a more "generous" scheduler in the framework without implementing a custom one?

PLINQ is already tuned to make maximum use of the hardware.

You can see this for yourself if you replace the Thread.Sleep call with something that actually uses the CPU (e.g., while (true) ;), and then watch your CPU usage in Task Manager. My expectation is that the 7 or 8 threads used by PLINQ in this example is all your machine can handle.

answered May 24 '17 at 13:51

Stephen Cleary

437,863
77
675
810

Agree with the first statement about efficiency as physically there are only X cores that can run X instructions at the same time. But, I presume there are some kernel/hardware optimisation that can interleave instructions, otherwise how could the PC run your instructions on all cores and yet run also other applications? Therefore virtually I could see X threads active and busy where X is higher than the number of processors. – user4388177 May 24 '17 at 13:53
@user4388177: [Preemptive multitasking](https://en.wikipedia.org/wiki/Preemption_(computing)). Like I said, sure, you can make it use more threads, but it won't run faster - it'll run slower due to context switches. – Stephen Cleary May 24 '17 at 13:55
1

@user4388177 You can't have more threads *active* than you have cores. You can have more threads, *sitting around accomplishing nothing* than you have cores. – Servy May 24 '17 at 13:56
I'm tempted to remove PLINQ from the example as it making it just confusing, I used it just to remove any delay in starting the next task, but has nothing to do with the experiment itself. – user4388177 May 24 '17 at 13:57
@StephenCleary Servy again, agree it won't be efficient, but my experiment was only to fire multiple tasks and having the OS try and run them in parallel, not about performance optimization. – user4388177 May 24 '17 at 13:58
@Servy, yes you can. Threads waiting for a signal are still considered active, and you can have a lot of them. – arbiter May 24 '17 at 14:03
@user4388177: You can just toss a lot of `Task.Run` tasks out there. But I'm not sure what hypothesis this experiment is intending to prove/disprove. – Stephen Cleary May 24 '17 at 14:05
@arbiter They're not *actively doing anything*. They're just sitting around doing nothing, because they can't run, because there's no CPU that can be running it. Like I said, you can have *lots* of threads, *sitting around doing nothing*, but your hardware will determine the number of threads that can be *actively doing anything* at any given point in time. You're using active to mean, "Wants to do something", not "is actually doing something". Those threads you create may all *want* to do work, but they won't be *able* to do work (at the same time). – Servy May 24 '17 at 14:08
@StephenCleary Copied from my other comment: I'm trying to tell the CLR to take more threads from the threadpool and run more tasks in parallel on them regardless of hardware/OS limitations. – user4388177 May 24 '17 at 14:10
@Servy, you trying to explain some common sense statements, of course, CPU cannot execute more threads than cores it have). But waiting thread is actually doing something, it is waiting. So it can be a reason why one's wanting to force higher parallelism. – arbiter May 24 '17 at 14:15
@arbiter Sure, if you want your work to get done more slowly, you might want to force more workers to be created than you have threads. Of course, if you just want your program to be less efficient and get less work done you could just as easily not ask it to do as much work in the first place and safe yourself some electricity. There are simply easier, and clearer, ways of intentionally making your program do less work, if that's actually what you want to happen, than by trying to force it to do more work than it's able to do and creating inefficiencies as a result. – Servy May 24 '17 at 14:18
@Servy, you missing the point. Actually when your workers depend on some external condition (network request, for example), increasing number of threads above number of cores can, actually, improve performance. It is not how most people (including myself) will use `AsParallel`, but I've seen such examples. After all ability to increase degree of parallelization exists in TPL for a reason. Not just because. – arbiter May 24 '17 at 14:24
1

@arbiter If you have network IO to do then the appropriate approach is to make the program asynchronous so that *you don't need multiple threads* to do the work in parallel, rather than to create a bunch of threads that are just going to sit around doing nothing, because you don't actually have any work for them to do. PLINQ also exists *specifically* for doing CPU bound work in parallel, and not for asynchronous non-CPU bound work, so it's the wrong tool for the job. Trying to use it, and trying to force it to create more threads than it knows is proper, is using it against its design. – Servy May 24 '17 at 14:27
@Servy, asynchrony is not a silver bullet. When working with network, good ol' synchronous code with multiple threads are often faster in real world situations that fancy async code. And I have to disagree, PLINQ is only about doing _anything_ on query in parallel. It is just a tool, and it is up to programmer to decide how to use it. – arbiter May 24 '17 at 14:47
@arbiter Asynchrony doesn't solve *all* problems, but it's *exactly* the right solution for dealing with problems that are *inherently* asynchronous, such as network IO. In such situations the operations are *inherently* asynchronous; you're simply choosing to create threads only to have them sit there doing nothing until the work is done, at which point you then immediately let them complete, after never having done anything of use. That's not helpful or productive. As for PLINQ, the name literally stands for "Parallel LINQ", so yes, it *is* about doing a query in parallel. – Servy May 24 '17 at 14:55
@StephenCleary I asked that specific question here: https://stackoverflow.com/questions/44161089/performance-of-multi-threading-exceeding-cores and apparently having more threads than cores can be useful (case by case, of course). – user4388177 May 25 '17 at 14:03

score 3 · Accepted Answer · answered Jun 05 '17 at 09:15

3

Useful link that explains it can be done with ThreadPool.SetMinThread:

https://gist.github.com/JonCole/e65411214030f0d823cb#file-threadpool-md

answered Jun 05 '17 at 09:15

user4388177

2,433
3
16
30

This solution wouldn't make your threads run in parallel, if you have less cores than simultaneously running threads. In general, this will lead you to performance degradation rather than performance gain – VMAtm Jun 09 '17 at 16:37
@VMAtm if threads are not executing active instructions on the CPU it will improve performances. Have you read the article? – user4388177 Jun 11 '17 at 11:39
Yes, I did, and I'm talking about situation thwn you have work for all the minimal number of threads, and that number is more than number of cores. You really should measure. – VMAtm Jun 11 '17 at 12:10
@VMAtm Indeed, is a matter of monitoring performances with different values and find the ideal setting. – user4388177 Jun 12 '17 at 11:37

score 0 · Answer 3 · answered May 24 '17 at 12:53

0

Try this: https://msdn.microsoft.com/en-us/library/system.threading.threadpool.setmaxthreads(v=vs.110).aspx

You can set the number of worker threads (first argument).

answered May 24 '17 at 12:53

Mikkel LP

71
6

1

@user4388177 the problem is that your code doesn't do anything except creating blocks using at least 3 different concurrency paradigms (Data parallelism with PLINQ, task parellelism with async/await, raw threads). Whatever problem you encounter is caused by the code itself – Panagiotis Kanavos May 24 '17 at 13:27
@PanagiotisKanavos PLINQ is irrelevant and the raw thread is only for monitoring. – user4388177 May 24 '17 at 13:39
1

@user4388177 no, it's *Very* relevant. It's the very thing that specifies the use of workers, partitioning and sets the number of workers. You can remove everything else from your code, it will still work the same. – Panagiotis Kanavos May 24 '17 at 13:47
@PanagiotisKanavos remove `AsParallel` and you get the same question (now that I added `await Task.Yield`). – user4388177 May 24 '17 at 13:49
@user4388177 No, you *don't* get the same question. You still have a rather nonsensical question that has lots of major problems in what it's trying to do, and still doesn't actually have any sensible problem that it's trying to solve. You're trying to figure out how to write code that accomplishes nothing accomplish it less effectively. On top of that, the mixing and matching of entirely unrelated tools causes major problems, and choosing to use an entirely different tool to generate the parallelism *radically* changes the question, rather than your assertion that it has no change. – Servy May 24 '17 at 13:54
@user4388177 Honestly it looks like you're just not ready to write multithreaded code yet. It's probably best to just stick to single threaded code at the moment. – Servy May 24 '17 at 13:55
@Servy looks like your comment is as pointless as your knowledge in this conversation. – user4388177 May 24 '17 at 14:00
@user4388177 The fact that you're choosing to ignore all of the people giving you good advice, and instead choosing to only listen to the people giving you incorrect information (simply because it's what you want to hear, despite it being incorrect and actively harmful) doesn't make my comment pointless. – Servy May 24 '17 at 14:02
@Servy wow, you managed to make a pointless comment about my comment to your pointless comment! That requires skills! – user4388177 May 25 '17 at 10:24
@user4388177 So why are you more interested in insulting people trying to help you solve your problem than you are learning from the advice you're being given as to what you're doing wrong and how you can fix it? Do you just not actually care about your programming problem, and just care about trolling people, or what? – Servy May 25 '17 at 13:11
@Servy I'm just having fun seeing your comments. Which by the way are unrelated to the question so your own last comment is even more hilarious. – user4388177 May 25 '17 at 13:36
@user4388177 So is your hypocrisy intentional, or unintentional? Are you constantly posting comments that are entirely unrelated to your own question in which you just insult people for trying to help you because you don't want people to help you, or because you find that just more entertaining? – Servy May 25 '17 at 13:38
@Servy I'm not insulting anyone, just mocking you. This is supposed to be a tech Q&A, you're just making unrelated comment, at least I can get a laugh out of it as I can't get any tech information from you apparently. I'm still open to discussion about the question with other people. – user4388177 May 25 '17 at 13:43
@user4388177 Mocking someone is a way of insulting them, so yes, you are. No, this is not a "teaching" QA, it's a repository of knowledge. I'm *not* just making unrelated comments; I'm helping you understand the problems with your code and how to fix them, and *you* are posting unrelated comments insulting me in response. That you're choosing to ignore all of the relevant technical information I'm presenting to you doesn't mean I'm not posting any. You've more than shown that you're not open to having a discussion, given that your response to people giving you solutions is to insult them. – Servy May 25 '17 at 13:46
@Servy OK, can you point me to the exact place where you left a useful technical comment? I'm happy to reply to it if I haven't already. – user4388177 May 25 '17 at 13:59
@user4388177 https://stackoverflow.com/questions/44158703/?noredirect=1#comment75337376_44160402 https://stackoverflow.com/questions/44158703/?noredirect=1#comment75337946_44160402 https://stackoverflow.com/questions/44158703/?noredirect=1#comment75338442_44160402 https://stackoverflow.com/questions/44158703/?noredirect=1#comment75338878_44160402 https://stackoverflow.com/questions/44158703/?noredirect=1#comment75340345_44160402 https://stackoverflow.com/questions/44158703/?noredirect=1#comment75337296_44159004 – Servy May 25 '17 at 14:07
"You can't have more threads active than you have cores." yes, you can: https://stackoverflow.com/questions/44161089/performance-of-multi-threading-exceeding-cores and it actually makes also sense to start more threads than cores in many cases. – user4388177 May 25 '17 at 14:10
"Sure, if you want your work to get done more slowly" the point of the example was not performances, just testing .NET internals. But yes, it can be more performant if some threads are busy but don't have instructions to execute, see the link in my previous comment. – user4388177 May 25 '17 at 14:12
For the next links, my question has nothing to do with async work (no IO/external resources, I specifically used Thread.Sleep to state that had nothing to do with the question) – user4388177 May 25 '17 at 14:13
Not even going to comment the last link as it makes no sense PLINQ has nothing to do with the question and I also removed it so you don't get confused. Happy @Servy? – user4388177 May 25 '17 at 14:14
@user4388177 You can have more threads than cores. You can't have more threads *doing anything* than you have cores, which is of course what the comment says. While there are situations where it makes sense to have more threads than cores, this is unquestionably not such a situation (performing portions of some larger operation in parallel) and most of the situations where it's done tend to simply be situations where a language/framework simply wasn't set up to solve the problem correctly. – Servy May 25 '17 at 14:19
@user4388177 As mentioned in one of those comments, if you just want to come up with a way of making your program run slower, without accomplishing anything else (which *is* occasionally a valid thing to do) there are *much* better ways of doing that. As it is, you're asking how to do something that's actively harmful, and that has no benefits. – Servy May 25 '17 at 14:21
@user4388177 I brought up IO as I was replying to a comment suggesting that that's a situation where creating more threads than you have cores is appropriate (it's not, as the comment explains). – Servy May 25 '17 at 14:22
@user4388177 You *asked a question about PLINQ*, so it does in fact have quite a lot to do with the question. You *specifically* asked how to affect PLINQ's thread creation algorithm. That you decided to completely change your question into an entirely different (but also not really sensible) question, just because you were being confused by the answer to that question, doesn't change that. – Servy May 25 '17 at 14:23
@Servy "this is unquestionably such a situation"? Because Thread.Sleep does a lot, sure :). – user4388177 May 25 '17 at 14:52
@Servy if the real content of the task is doing a mix of lock/sleep/io/actual work then more threads is beneficial as stated in the other topic. – user4388177 May 25 '17 at 14:53
@user4388177 If the sleep is a placeholder for actual CPU bound work, then yes, it is. If it's not, and your real code just wants to sleep, then there's no reason to create multiple threads, just to have them sit around doing nothing. You don't need *any* threads to do nothing for a fixed period of time. Use a timer if you want to run some code after a fixed period of time and you'll use *no* threads to accomplish the same task much more effectively. – Servy May 25 '17 at 14:54
@Servy this is why I prefer to joke, because if you keep saying nonsense things, the conversation is pointless from an educational perspective... PLINQ has nothing to do with it and I just removed one method call which doesn't affect the meaning. That said I'll go back to mocking as playing your game was again... guess what? pointless... – user4388177 May 25 '17 at 14:55
@Servy real code does both, you're just trying to be right now, not being constructive. Sorry, I did a bit mistake taking you seriously about wanting to turn it into a constructive conversation, never again :). – user4388177 May 25 '17 at 14:56
@user4388177 But you [just stated](https://stackoverflow.com/questions/44158703/tpl-force-higher-parallelism/44159004?noredirect=1#comment75379434_44159004) that you're *not* doing any IO. Now you're saying that you *are* doing IO? Which is it? If you *are* just doing a bunch of IO and synchronization, then no, you don't need additional threads, and the structure of your code is quite flawed (as you're using tools specifically designed for CPU bound work that doesn't require any synchronization between sub-operations). – Servy May 25 '17 at 14:56
IO is not the only thing that is not active work. I said it's a mixture o both in real code, active instructions and sleep/lock/IO etc... Why am I still typing? XD – user4388177 May 25 '17 at 14:57
Poor @MikkelLP, when he logs back in he will probably commit harakiri :). – user4388177 May 25 '17 at 15:00
@user4388177 I'm not saying nonsense things. I'm telling you how to do what you're asking, but much more effectively. You on the other hand are constantly contradicting yourself as to what you're asking. If you say you want one thing, then get an answer, and *then* say you want something completely different, you can't then say that people aren't being constructive for answering the question you asked. Using PLINQ *doesn't* have nothing to do with what you're doing. Removing it *completely* changes your question, what it's doing, and what the answer is. – Servy May 25 '17 at 15:02
@user4388177 No, you *didn't* say that your code was a mixture of IO and other code. When people have assumed you just had CPU bound work you said you didn't, when people assumed you had IO bound work you've said you don't, you've stated that all your code is really doing is sleeping, then when people responded under that assumption you said that's absurd and not what you're actually asking about. If you're going to constantly change the question you're asking, you're *going* to cause problems for your question, and simply won't get any meaningful answer. – Servy May 25 '17 at 15:04
@user4388177 But hey, at least you're admitting that you're just here to troll people, rather than have a constructive technical discussion. Now that we all know that you don't care at all about getting an answer, just trolling people who are trying to help you, we can safely ignore you. – Servy May 25 '17 at 15:05
@Servy I would recommend changing job, you would make a great fantasy novelist; unless... you already are! – user4388177 May 25 '17 at 15:25
Please tell me it wasn't you downvoting my other answer XD so I can keep a crumb of respect, that makes me really laugh. – user4388177 May 25 '17 at 15:30

score 0 · Answer 4 · answered May 24 '17 at 13:23

0

Use WithDegreeOfParallelism extension:

Enumerable.Range(0, 100).AsParallel().WithDegreeOfParallelism(x).Select(...

answered May 24 '17 at 13:23

arbiter

9,447
1
32
43

1

Which won't help at all. The OP's code is generating sleep statements in a PLINQ query. Doing nothing faster is still doing nothing. In a *real* example, a higher DOP would hurt performance as PLINQ would try to use more threads than there are cores to process the (large) input data, resulting in thread switching – Panagiotis Kanavos May 24 '17 at 13:25
Completely disagree. It is totally depend on the code which is executed in parallel. For pure-math you statement makes sense, if any wait for external resources involved, it is not (it is where async shines sometimes, but that's different story). – arbiter May 24 '17 at 13:39
@arbiter the parallelism in invoking the async lambdas will not make any difference, as soon as they hit `await` the methods return. Actually until `await` it executes synchronously so it makes sense either to do as you suggested or add `await`. – user4388177 May 24 '17 at 13:41
1

Completely disagree with your disagreement. TPL covers multiple paradigms. PLINQ was created to make data parallelism easier, not math. For that scenario, using more threads than can run is pointless. It's *NOT* meant for IO parallelism, asynchronous programming, or performing only a few CPU-heavy operations – Panagiotis Kanavos May 24 '17 at 13:41
1

@arbiter that's why PLINQ and Parallel.For/Foreach *partition* the data before passing them to the worker tasks, to avoid wasting CPU time in thread switching. Partitioning isn't very meaningful if you have a lot of IO. – Panagiotis Kanavos May 24 '17 at 13:42
@user4388177 there's nothing wrong with PLINQ. Try a realistic example. How does execution time change with an *actual* data load if you use a DOP that's larger than the number of cores? You'll notice that it *does* rise as you add more workers – Panagiotis Kanavos May 24 '17 at 13:45
@PanagiotisKanavos as `Thread.Sleep` is blocking, that line doesn't return all the tasks until the last one finally finishes. @arbiter had a good point about that. – user4388177 May 24 '17 at 13:48
@PanagiotisKanavos, easy. Try another realistic example. You get some value in worker, modify and then send to another machine via network call. It is not how I would implement such task, but it is still possible. So, how I mentioned before, it is totally depend on the code. – arbiter May 24 '17 at 13:57

TPL force higher parallelism

4 Answers4