Performance deviation for a small method

Question

We have an OSGi container with a lot of products running inside, one of them being our product.
We have some performance tests running and there is this weird problem that, every OSGi container restart will result in a performance deviation for some of our tests up to 400%.
Through some testing and things I was able to track this down to this method:

public static Method getMethodForSlotKey(Class<?> cls, String slotKey, String methodType) {

    Method[] methods = cls.getMethods();

    if (methods != null && methods.length > 0) {
        for (Method method : methods) {
            String methName = method.getName();
            if (methName.startsWith(methodType)) {

                IDataAnnotation annot = method.getAnnotation(IDataAnnotation.class);
                if (annot != null) {
                    String annotSlotKey = annot.SlotKey();
                    if (annotSlotKey != null && annotSlotKey.equals(slotKey)) {
                        Class<?>[] paramTypes = method.getParameterTypes();
                        // for now, check length == 1 for setter and 0 for getter.
                        int len = SET_TXT.equals(methodType) ? 1 : 0;
                        if (paramTypes != null && paramTypes.length == len) {
                            return method;
                        }
                    }
                }
            }
        }
    }
    return null;
}

This method mainly does reflection and string comparison.

Now, what I did is to cache the results of this method and instantly our deviation goes down to 10 - 20%. Of course, this method is called often, so that there is am improvement is obvious.

Still I don't understand why the non-cached version has such a high deviation with the only difference being a OSGi / JVM restart? What exactly may happen during the restart? Are there any known performance issues for different classloaders for instance? Is it possible that in the OSGi environment libraries will be loaded in a different order between the restarts?

I am searching for a clue here for this to make sense.

UPDATE

It turns out that this call:

Method[] methods = cls.getMethods();

is causing the deviation. I still don't understand it, so if anyone does I'd be happy to hear about it.

Answer 1

It seems like this is another case of the intricate JIT, which seems to constantly change and evolve with time. This answer may shed some light on your issue: Java: what is JITC's reflection inflation?

I believe you are observing the aftereffects of how JIT optimizes your reflection calls. For a while (the first X invocations) would be slow until JIT decides to optimize your method.

Something you can try - to test this theory - is to modify your code and put the offending call in a dummy loop and execute it a number of times on the very first call (make sure you do something with the result from the call to force java to not optimize away the call completely, if it figures it has no consequences). If this results in a very long execution time for the first use and then very low variance on the subsequent calls, then the theory sounds correct.

Answer 2

I suspect it is because your reboot of your server / OSGi container is clearing the cached Method[]* in the class(es) you are interrogating. The cache will be cleared to ensure correct operation in the situation where the reloaded class has a different public API (i.e. different methods) than the old one with the same name (this could happen e.g. if you load a newer version)

When you implement your cache of your 'wrapper' method, you are circumventing this and introducing an assumption - that the reloaded classes will have the same methods that the previously loaded classes with the same name had. Probably quite a safe assumption for most cases, but not one you would want core Java to make.

The source code for the (OpenJDK) Class class is here. If you look through it, you will see that getMethods() calls privateGetPublicMethods(). If the cached publicMethods array* is null, or is made null by clearCachesOnClassRedfinition(), then the class will have to go through the relatively intensive process of finding all the public methods in the class and any super classes it inherits from and interfaces it implements. This is then cached and the speed of getMethods() will return to normal.

---

*strictly, the cached value is a soft reference to an array.