2

Let's say I have a 6-core machine with 12MB cache. I use it for a server application that has a few gigabytes of heap (much of it 2nd level Hibernate cache).

I noticed that most of the time I have a handful threads actively serving client requests (burning CPU and talking to DB), as well as about 30-50 more threads that are only doing ol' good synchronous network IO with client.

As I am learning about Java memory model, I am wondering if this can impact performance. Does context switching for one of the many network IO threads ruin thread/CPU cache of the "active" threads? Is this level of concurrency harmful in itself (memory cache aside)?

Does it really matter, given how small the CPU cache is in relation to the whole application memory? How can I determine where the boundary is?

Charles
  • 50,943
  • 13
  • 104
  • 142
Konrad Garus
  • 53,145
  • 43
  • 157
  • 230
  • 1
    On a related note, in order to minimise context switching have you considered using async IO? http://stackoverflow.com/questions/2794535/linux-and-i-o-completion-ports – Slugart Apr 03 '12 at 09:42
  • @Slugart This question is actually part of research for using async IO. Trying to figure out if I may be already affected (and whether I get impact of the memmory model right) and whether it's worth it. – Konrad Garus Apr 03 '12 at 09:45
  • in theory it is more efficient as PeterLawrey mentioned the context switching will occur. In practice it is up to you to measure the difference in your scenario. – Slugart Apr 03 '12 at 09:47

2 Answers2

3

Does context switching for one of the many network IO threads ruin thread/CPU cache of the "active" threads?

There are no hard guarantees, but my gut feeling is that, in practice, CPU caches get updated by order(s) of magnitude more frequently than the scheduler switches threads. If this is the case, then the cache update required by thread context switching is extremely small overhead compared to the cache updates that are going on already.

How can I determine where the boundary is?

By experimenting and measuring (profiling), as is the case with most performance-related questions. Simple theoretical approximations won't work because the reality of what's really going on in a modern PC is more complex than any reasonable approximation.

Joonas Pulakka
  • 36,252
  • 29
  • 106
  • 169
2

There will be an impact and you can get some jitter when you have more active threads than cpus. However, the jitter is usually considered to be relatively small 10 to 100 micro-seconds.

You can have the thread competing for cache resources, but there is not much you can do about it except reduce the amount of active data each thread uses. This can reduce your cpu load by a factor of 2x or more, but involves alot of work, possibly a complete re-write (including some JDK and third party libraries) and it may yeild little benefit. e.g. how much do you need the cpu load to be lower.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130