Look at it like this:
It is certainly not desirable to cache all Integer values. For, this would mean that when a JVM starts up, it'd have to make more than 4 billion Integer objects which would most likely not fit in the memories of contemporary computers.
Hence, one must decide to cache some smaller number of Integers (if at all). This number should be small enough so that no memory usage or slower startup is noticeable. OTOH, it should cover as most cases as possible. Not having read any study concerning this, but from experience we can say with confidence that very small integers are most often used.
Thus, the decision to cache 128 numbers is completly arbitrary, yet it still makes sense. It could as well be 30 or 300. But certainly not a million.
How does this help performance? Well, it sppeds up autoboxing for small numbers, because one does not have to construct an Integer, but rather pick one from the cache (which is most likely an Integer[128] array) with a single memroy access. At the same time, it is well known that many Integers are short-lived. Use of pre-allocated objects that need not be garbage collected takes away stress from the GC.