Optimising Java code for fast response

Question

I have a multithreaded Java application that uses several threads that are CPU intensive to gather information. Once every few minutes, a result is found that requires handling by another thread of the program. The found result is added to a list and the other relevant thread is notified (using Lock and Condition), after which it handles the found information. I need the time delay for passing this information from thread to thread to be as small as possible. When measuring the time between wake-up and notify using System.currentTimeMillis(), the delay is usually smaller than 5 ms, and most often less than or equal to 1 ms. Sometimes, the delay is larger (10-20ms). Since milliseconds are macro-units when it comes to computers, I would think that a delay that is reliably smaller than 1ms should be possible, and it would benefit my application.

Do you have any idea what the cause of the larger delays can be, or how I can find out where to look? Could it be Garbage Collection? Or are several milliseconds of variation for thread wakeup considered normal?

I am using Java version 1.8.0 on a Linux Ubuntu virtual private server.

An example of the design of the program is attached. Running this does not simulate the delays as observed by my production program correctly. The 'actual' program uses a lot of memory, CPU and only transmits a bit of info once every few minutes. I tried but failed in simulating this simply.

Thank you.

import java.util.concurrent.locks.ReentrantLock;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.Condition;
import java.util.List;
import java.util.ArrayList;
import java.util.Random;

public class Example {
    public static void main(String[] args) {
        startInfoThreads();
        startWatcherThread();
    }

    private static Lock lock = new ReentrantLock();
    private static Condition condition = lock.newCondition();
    private static List<Long> infoList = new ArrayList<>();

    private static void startWatcherThread () {
        Thread t = new Thread () {
            @Override
            public void run () {
                while (true) {
                    // Waiting for results...
                    try {
                        lock.lock();
                        while (infoList.size() == 0) {
                            try {
                                condition.await();
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            }
                        }
                        long delta = System.currentTimeMillis() - infoList.remove(0);
                        if (delta > 0)
                            System.out.println("Time for waking up: " + delta);
                    } finally {
                        lock.unlock();
                    }
                    // Do something with info
                }
            }
        };
        t.start();
    }

    private static void startInfoThreads () {
        for (int i = 0; i < 14; i++) {
            Thread t = new Thread() {
                @Override
                public void run() {
                    Random r = new Random();
                    while (true) {
                        // Gather info, 'hits' about once every few minutes!
                        boolean infoRandomlyFound = r.nextInt(100) >= 99;
                        if (infoRandomlyFound) {
                            try {
                                lock.lock();
                                infoList.add(System.currentTimeMillis());
                                condition.signal();
                            } finally {
                                lock.unlock();
                            }
                        }
                    }
                }
            };
            t.start();
        }
    }
}

Answer 1

System.currentTimeMillis() can be affected by clock drift and usually have a granularity of ~10ms.

To measure elapsed time you should always use System.nanoTime() as it guarantees accuracy.

Answer 2

It probably will not speed up your process but using a BlockingQueue would certainly make the code clearer.

Also note the Thread.sleep for when there is no info.

final BlockingQueue<Long> queue = new ArrayBlockingQueue<>(10);

private void startWatcherThread() {
    Thread t = new Thread() {
        @Override
        public void run() {
            while (true) {
                // Waiting for results...
                try {
                    Long polled = queue.poll(1, TimeUnit.SECONDS);
                    // Do something with info
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    };
    t.start();
}

private void startInfoThreads() {
    for (int i = 0; i < 14; i++) {
        Thread t = new Thread() {
            @Override
            public void run() {
                Random r = new Random();
                while (true) {
                    // Gather info, 'hits' about once every few minutes!
                    boolean infoRandomlyFound = r.nextInt(100) >= 99;
                    if (infoRandomlyFound) {
                        queue.put(System.currentTimeMillis());
                    } else {
                        try {
                            Thread.sleep(1);
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                    }
                }
            }
        };
        t.start();
    }
}

private void test() {
    startInfoThreads();
    startWatcherThread();

}