2

According to JEP 425,

virtual threads can significantly improve application throughput when the number of concurrent tasks is high (more than a few thousand), and the workload is not CPU-bound

Why are virtual threads not helpful when the thread count is much lower than a few thousand? If I have 50 concurrent I/O-bound tasks, will I not significantly reduce CPU load by using virtual threads to eliminate heavyweight OS thread context switching?

Andrew Parks
  • 6,358
  • 2
  • 12
  • 27
  • Depends on the problem. You need to understand concurrency and parallelism: https://www.geeksforgeeks.org/difference-between-concurrency-and-parallelism/#:~:text=Difference%20between%20Concurrency%20and%20Parallelism%3A%2D&text=Concurrency%20is%20the%20task%20of,of%20running%20multiple%20computations%20simultaneously. – duffymo Sep 23 '22 at 17:14
  • I suppose this hinges on what you consider significant. I would interpret this as saying that the benefits are most clear with high number of threads (where per-thread memory usage becomes a bigger concern), not that there are no benefits with fewer threads. – Tim Moore Sep 23 '22 at 22:26

2 Answers2

2

In short: in case of 50 concurrent tasks you won't gain much by using virtual threads. They won't make things worse either - you'll just hardly notice any difference. But when the number of native threads is high, things start to change.

How high? A typical application can sustain only a several thousand threads running simultaneously (an empirical observation). Beyond that you'll most likely run out of RAM (thread stacks will take tens of gigabytes of memory), plus constant context switching will slow down your application.

Virtual threads are switched in user space and their stacks are tiny (hundreds of bytes). Thus, your application will likely survive millions of them.

So, unless you have a need to big number of clients simultaneously (by big I mean hundreds and thousands), you won't benefit much from using virtual threads. Those are for monstrous I/O bound workloads. But I suggest try and measure, of course.

raiks
  • 1,270
  • 1
  • 15
  • 12
  • I guess the answer comes down to: 50MB isn't very much memory to be concerned about if you only have 50 threads and use 1MB per thread for each thread stack. And after that, I guess the context switching overhead is really not great enough to notice for only 50 threads. – Andrew Parks Oct 07 '22 at 14:02
1

The default number of threads in one of the most popular servers (tomcat) is 200. Any number of threads above the core count will result in context switches, this includes threads that are not owned by your process.

There is overhead even for idle (sleeping) threads. The scheduler has to ping them from time to time to see if any of them needs to be awakened. This means that it is always a good idea to keep the thread count low and as close to the number of cores as possible.

No harm in using VT as mentioned by the other answers you should experiment and see how they behave in your app but don't expect any change for that tiny load

Borislav Stoilov
  • 3,247
  • 2
  • 21
  • 46
  • Thanks. I guess I should write my own tests to discover exactly how much of an overhead really is involved with context switches. Right now, all I have is a vague notion that there is some kind of overhead. – Andrew Parks Oct 26 '22 at 20:37