High-priority threads in Java

Question

I am currently working on performance of a distributed application. I am targeting a network component. Currently for each connection there is a dedicated thread which handles a socket in blocking mode. My goal is to reduce number of threads(without reducing performance) and if possible improve performance.

I redesigned the network component to use async communication and am trying to use 1 to 2 threads for entire network processing. I did a simple test where I wrote in a loop from one node and read on another, This was to test max nw thread capability and I found that my busy loop implementation was consuming 100% cpu and was getting much more operations per sec then we require. So I integrated this busy loop implementation in the existing application.

Problem I found is that other threads are not allowing these async nw threads to aquire full cpu, even though I have a 8 core system and we are not using more than 400% cpu. Basically being a C programmer I would have solved this by binding my nw thread on a core and raising its scheduling priority, so that other threads can still run on other core. I am not able to do similar in Java. There are conflicting comments on Java thread priority. Also I do not want to reduce the priority of other threads as it may have its own side affect.

How would you solve this problem?

Answer 1

I have a library to supports thread affinity in Java on Linux and Windows. https://github.com/peter-lawrey/Java-Thread-Affinity

If you isolate the CPUs you can ensure the CPUs you assign will not be using for anything else (other than non-maskable interrupts) This works best in Linux AFAIK.

---

You can get lower latency results if you use busy waiting with non-blocking NIO than blocking IO. The later works best under load, at lower loads the latency can increase.

You might find this library interesting https://github.com/peter-lawrey/Java-Chronicle it allows you to persist millions of messages per second, optionally to a second process.

BTW: Thread priority is just a hint, the OS is free to ignore it (and often does)

---

A simple example comparing warm vs cold code. All it does is copy an array repeatedly and time it. Once the code and data has warmed you wouldn't expect it to slow, but all it takes is a 10 ms delay even on a quite machine to slow the time it takes to do the copy significantly.

public static void main(String... args) throws InterruptedException {
    int[] from = new int[60000], to = new int[60000];
    for (int i = 0; i < 10; i++)
        copy(from, to); // warm up
    for (int i = 0; i < 10; i++) {
        long start = System.nanoTime();
        copy(from, to);
        long time = System.nanoTime() - start;
        System.out.printf("Warm copy %,d us%n", time / 1000);
    }
    for (int i = 0; i < 10; i++) {
        Thread.sleep(10);
        long start = System.nanoTime();
        copy(from, to);
        long time = System.nanoTime() - start;
        System.out.printf("Cold copy %,d us%n", time / 1000);
    }
}

private static void copy(int[] a, int[] b) {
    for (int i = 0, len = a.length; i < len; i++)
        b[i] = a[i];
}

prints

Warm copy 20 us
Warm copy 20 us
Warm copy 19 us
Warm copy 23 us
Warm copy 20 us
Warm copy 20 us
Cold copy 100 us
Cold copy 80 us
Cold copy 89 us
Cold copy 92 us
Cold copy 80 us
Cold copy 112 us

Answer 2

This really smacks of premature optimization to me. You have an 8 core system and are only using 400% CPU. What makes you think that this is not a textbook example of IO bound program? What makes you think that you've not maxed out your network IO chain?

@Peter knows his stuff and I'm sure you can hack processor affinity and force your critical threads to a single CPU but the question is will it make your program run any faster? I sincerely doubt it. The model Java VM is very smart about thread scheduling and I suggest that it is doing its job appropriately. Unless you have very good evidence to the contrary, I would let it handle the scheduling. Even priorities mean very little if most of the threads are waiting for IO.

Also, what makes you think that reducing the number of threads is somehow better. This moves a lot of code from native land (i.e. thread multiplexing) into Java land (i.e. NIO code). If you are talking about 1000s of threads then I'd agree but even 100s of threads should be an efficient way to handle the connections.

I've done a ton of thread programming for more than two decades and I've never had to force thread affinity. Certainly sizing thread-pools and making good decisions of where to apply thread-pools versus dedicated threads is an art but force the VM to schedule the threads the way you think they should be is just not a good use of your time. Spent some time with a profiler to find out where your program is spending its time would be a better investment IMHO.