Did some source code digging and coding myself to figure this out, and here's what I've found out:
Java's 'Method' class has a member variable 'methodAccessor' of type 'MethodAccessor' which is an interface with a method 'invoke', similar to Method's invoke. Methods's invoke delegates to methodAccessor's invoke.
If inflation is enabled (noInflation is false) this accessor points to an implementation which uses JNI to run this Java method (I think using api's like GetObjectClass, GetMethodID and Call*Method). This is like duel dispatching, and execution with JNI is slow due to this and other reasons.
( What makes JNI calls slow? )
After 15 executions of a method through reflection ('15' is default and can be changed) and with noInflation false, the JNI based accessor creates a class on the fly (the name is dynamically generated, e.g. say 'GeneratedMethodAccessor1') which also has the invoke method. Now, within this 'invoke' method, it casts the first 'obj' argument to its corresponding class, and then calls the target method on it. It then creates an instance of this class, and changes the methodAccessor settings such that every execution of the method henceforth is delegated to this instance instead of JNI accessor. This is called inflation.
Because this instance is of a Java class which delegates to a Java object, the delegation henceforth is a normal Java delegation. It never goes to JNI and hence saves that overhead, plus JITC can perform other optimization on it due to which it becomes efficient.
The downside is, if a lot of methods are inflated in this manner, their classes occupy permgen space and can possibly cause out of memory error.
For details, see:
http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/classes/sun/reflect/ReflectionFactory.java
http://java.sun.com/docs/books/jni/html/fldmeth.html
http://anshuiitk.blogspot.com/2010/11/excessive-full-garbage-collection.html