Efficient way of getting thread CPU time using JMX

Question

I'm currently getting the total thread CPU time using JMX in the following manner:

private long calculateTotalThreadCpuTime(ThreadMXBean thread) {

    long totalTime = 0l;

    for (ThreadInfo threadInfo : thread.dumpAllThreads(false, false))
        totalTime += thread.getThreadCpuTime(threadInfo.getThreadId());

    return totalTime;
}

As the ThreadMXBean is actually a remote proxy, performance is dreadful, in the order of magnitude of seconds for this actual method call.

Is there a faster way of doing this?

---

Update: I'm using this for performance monitoring. The measurements were both 'wall clock' time and JProfiler, which showed about 85% of my time spent in this method. I do have some other MXBean calls ( Runtime, Memory, GC ), but they are much cheaper. Most likely because every call to thread.getThreadCpuTime is a remote one.

Update 2: JProfiler screenshot showing the performance problems.

Answer 1

If you are willing to use non-standard APIs, you can cast OperatingSystemMXBean to com.sun.management.OperatingSystemMXBean and invoke getProcessCpuTime(), as described in Using a Sun internal class to get JVM CPU time on David Robert Nadeau's blog.

Answer 2

It must be the remote nature of your calls that is the problem.

I recently did a spike on asking a thread for CPU time uses and it was incredibly faster. In the context of a web application request it was nearly immeasurable. If one went into a hard loop it would costs you but typically you want it at the start of an operation and again at the end.

Answer 3

The cause of the poor performance is that each call of thread.getThreadCpuTime() will (in your case of a remote proxy) most probably result in an RMI/Remote JMX call which of course is expensive, especially when the message is actually sent via TCP (and maybe even outside localhost). (see JMX 1.4 spec, chapter 7.3)

Although JMX defines multiple attribute retrieval at once this will not help you out here because thread.getThreadCpuTime(long) is not an JMX attribute but a remote method invocation (and additionally, the ThreadInfo object on which the methods is invoked differs by each call)

Unfortunately, the JMX 1.4 spec does not define something like "invoke multiple methods of multiple objects and return their return values at all". So beside concurrent invocation (which will save time but will not reduce the effort) of these calls I am not aware of any (compatible) optimizations here.

Answer 4

Optimisations:

• invoke getThreadCPUTime inside a thread pool, since it seems to be network-bound;
• whenever a thread is found to be in Thread.STATE.TERMINATED, keep its name in a Map and skip querying the next time.

Answer 5

Can you talk more about how you're planning on using this information? You can get thread CPU times using a JVMTI based agent, which should perform slightly better than JMX. How have you measured the overhead of the JMX approach that lead you to the conclusion of 'dreadful' performance?