12

Let's say I'm using Java 11 javac, but I'm using the --source and --target options set to 1.8 so that my source code will be considered Java 8 and the output .class files will be compatible with Java 8. My goal is to produce .class files that can run on a Java 8 JVM.

And let's say I have the following Java 8 code I'm compiling:

import java.nio.ByteBuffer;

…

ByteBuffer byteBuffer = …; //init somehow
byteBuffer.flip(); //what ends up in the `.class` file?

The question is: what should Java 11 javac put in the .class file to link the byteBuffer.flip() method call? Before you answer, consider this:

  • Neither Java 8 ByteBuffer nor Java 11 ByteBuffer declare the flip() method at the ByteBuffer level.
  • ByteBuffer is a direct subclass of Buffer. There is both a Java 8 Buffer.flip() and a Java 11 Buffer.flip() declared in the Buffer API for both versions.
  • In the Java 8 source code, there is no ByteBuffer.flip() method.
  • But in the Java 11 source code, ByteBuffer overrides the Buffer.flip() method like the following. The purpose apparently was to use covariance so that in Java 11 ByteBuffer.flip() would conveniently return a ByteBuffer instead of a Buffer.
    @Override
    public ByteBuffer flip() {
        super.flip();
        return this;
    }
    

So to restate the question: Should Java 11 javac, with the --source and --target options set to 1.8, generate a .class file that links to Buffer.flip() or to ByteBuffer.flip()? If it is the former, then how does it know not to include ByteBuffer.flip() instead, as the (Java 8) code clearly references ByteBuffer.flip() and the (Java 11) compiler sees that there is a ByteBuffer.flip() method in the runtime? But if it is the latter, then how can I ever know that my 100% correct Java 8 compatible source code, when compiled using Java 11, will run on a Java 8 JRE even if I use the --source and --target options to indicate Java 8? (Note that OpenJDK 11.0.5 seems to choose the latter option. But which is correct?)

(Note that I'm using the word "link" loosely; I'm not currently well-versed in what bytecode is generated. All I know is that the class file has come reference somehow to Buffer.flip() or ByteBuffer.flip(); and if this method can't be found at runtime, the JVM will throw an exception such as: java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;.)

As a bonus question, I wonder whether using the --release option set for Java 8 would change the answer. But note that I can't use the --release option (equivalent to the Maven <release> compiler plugin option) because I want my Maven project to be buildable with both Java 8 and Java 11.

John Kugelman
  • 349,597
  • 67
  • 533
  • 578
Garret Wilson
  • 18,219
  • 30
  • 144
  • 272
  • You keep saying "link" but talking about things that aren't linking. It'll use `invokevirtual` and the standard rules apply. – chrylis -cautiouslyoptimistic- Nov 17 '20 at 03:49
  • Thanks for pointing out the terminology. I'm using "link" loosely, because I don't know what goes in the `.class` file. But it must reference the method somehow. Does it reference the method in terms of `Buffer` or `ByteBuffer`? I clarified the question a bit. When I see an error message such as `java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;`, does that mean the `.class` file contained an explicit reference to `ByteBuffer.flip()`? Or did the `.class` file contain some sort of more general reference, and the JRE merely inferred `ByteBuffer.flip()`? – Garret Wilson Nov 17 '20 at 04:47
  • 1
    Just a small addition. There is a [small library](https://github.com/headius/backport9/blob/master/src/main/java/com/headius/backport9/buffer/Buffers.java) that was created for this kind of problems with `ByteBuffer` on Java 9+. – ZhekaKozlov Nov 19 '20 at 05:44

1 Answers1

18

If we take the following code and compile with Java 8 and with Java 11, we get the following bytecode, as seen when running javap -c MyClass.class.

Java Source code

ByteBuffer byteBuffer = ByteBuffer.allocate(64);
byteBuffer.flip();

Java 8 bytecode

 0: bipush        64
 2: invokestatic  #19   // Method java/nio/ByteBuffer.allocate:(I)Ljava/nio/ByteBuffer;
 5: astore_1
 6: aload_1
 7: invokevirtual #25   // Method java/nio/ByteBuffer.flip:()Ljava/nio/Buffer;
10: pop

Java 11 bytecode

 0: bipush        64
 2: invokestatic  #19   // Method java/nio/ByteBuffer.allocate:(I)Ljava/nio/ByteBuffer;
 5: astore_1
 6: aload_1
 7: invokevirtual #25   // Method java/nio/ByteBuffer.flip:()Ljava/nio/ByteBuffer;
10: pop

As you can see, both of them "link" to the flip() method of ByteBuffer, even though the method isn't declared there for Java 8.

However, at the bytecode level, method signatures include the return type. This means that the JVM supports languages where you can overload methods that differ only in return type, even though Java doesn't support that.

The Java 11 version of the method has a different return type, which can be seen in the "linked" method, following the (), where Java 8 shows return type as Ljava/nio/Buffer; and Java 11 shows return type as Ljava/nio/ByteBuffer;.

When you take the code that was compiled against the Java 11 Runtime Library, and you try running it on Java 8, you get Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;

This is why you should always specify the bootstrap class path to point to a Java Runtime Library matching the target Java version. When you compile using Java 11's javac with options -source 8 -target 8, it will actually warn you about that:

warning: [options] bootstrap class path not set in conjunction with -source 8
1 warning

This is why they implemented the newer --release <release> option to replace -source and -target. If you compile with --release 8, the generated .class file will run without error on Java 8.


UPDATE

You don't need Java 8 installed to use option --release 8. The Java 11 installation knows what the methods of the Java 8 Runtime Library were.

The --release option was implemented in Java 9 as a result of JEP 247: Compile for Older Platform Versions, which says:

For JDK N and --release M, M < N, signature data of the documented APIs of release M of the platform is needed. This data is stored in the $JDK_ROOT/lib/ct.sym file, which is similar, but not the same, as the file of the same name in JDK 8. The ct.sym file is a ZIP file containing stripped-down class files corresponding to class files from the target platform versions.

Andreas
  • 154,647
  • 11
  • 152
  • 247
  • This is all very interesting, thank you. In my actual use case I'm using Maven. I wonder how I would set the "bootstrap class path" in Maven? I've never seen the warning you mention. Perhaps I should throw in the towel and use the Maven compiler plugin `` option, i.e. the `--release` javac option, which would require the project to be built with Java 11 but the artifacts would be compatible with Java 8. So you're saying in this case `--release` would cause this to "link" to `Buffer.flip()`? But how is that, if the bytecode is the same (as it seems to be in your example above)? – Garret Wilson Nov 17 '20 at 04:56
  • Huh. Did not know about the variant-return business. – chrylis -cautiouslyoptimistic- Nov 17 '20 at 04:58
  • 1
    @GarretWilson The bytecode is not the same, because Java 8 links to `Buffer ByteBuffer.flip()` and Java 11 links to `ByteBuffer ByteBuffer.flip()`, i.e. both link to `ByteBuffer.flip()` but with *different return types*. There is no link to `Buffer.flip()` in the bytecode of either version. – Andreas Nov 17 '20 at 04:59
  • @Andreas, oh, are you saying that the `#19` and the `#25` above are actually references to some method lookup table or something that have the full names you mention in the comments? – Garret Wilson Nov 17 '20 at 05:00
  • 2
    @GarretWilson #19 and #25 are references to entries in the "constants" table of the `.class` file. In the class file, they are basically those strings you see commented after the constant number. The JVM then resolves those strings (method signatures) against actual methods when the class is loaded. – Andreas Nov 17 '20 at 05:07
  • @Andreas thanks this is super useful. But the last thing I'm doubtful about: using the `--release 8` option, what would Java 11 stick in this lookup table? And how would it know to use the Java 8 version and not the Java 11 version? Surely Java 11 doesn't keep some sort of historical reference of all the API changes for all the Java versions, does it? – Garret Wilson Nov 17 '20 at 05:12
  • @chrylis-cautiouslyoptimistic- I've had fun looking into how the Java compiler has to handle the differences between the Java language and the bytecode instructions, using "synthetic" / "bridge" methods. E.g. at the bytecode level, there is no such thing as *inner*, *anonymous*, and *local* classes, and `private` members can only be accessed by the class itself, but at the Java level, `private` members are accessible to all classes inside the same top-level class. In Java, method `Number foo()` can be overridden in subclass by `Integer foo()`, but you can't do that at the bytecode level. – Andreas Nov 17 '20 at 05:13
  • 1
    @GarretWilson It does. The `--release` option was implemented as a result of [JEP 247: Compile for Older Platform Versions](https://openjdk.java.net/jeps/247), which says: *"For JDK N and `--release` M, M < N, signature data of the documented APIs of release M of the platform is needed. This data is stored in the `$JDK_ROOT/lib/ct.sym` file, which is similar, but not the same, as the file of the same name in JDK 8. The `ct.sym` file is a ZIP file containing stripped-down class files corresponding to class files from the target platform versions."* – Andreas Nov 17 '20 at 05:18
  • Wow, this is all great info. Thank you both so much. I'm going to digest this a bit and see if any more doubts pop up. – Garret Wilson Nov 17 '20 at 05:19
  • @Andreas If you really want to have fun, start playing with Groovy AST transforms. You can directly manipulate a lot of the details that are available in the JVM but not in Java. – chrylis -cautiouslyoptimistic- Nov 17 '20 at 06:36
  • I would like to add a bounty to reward this answer, but the "start a bounty" link isn't showing up. Do I need to unselect this answer first, or is this some sort of weird Stack Overflow behavior I wasn't aware of? – Garret Wilson Nov 18 '20 at 16:16
  • @GarretWilson You put up a bounty to entice people to answer, because you haven't gotten an (acceptable) answer yet. If you already accepted an answer, and hence closed the question, that option is of course no longer available. --- *FYI:* You don't need to give me a bounty, I'm not doing this for the rep. Of course, if you decided that you need/want a better answer than this, then un-accept the answer and put up a bounty. – Andreas Nov 18 '20 at 17:21
  • The bounty thing should normally show up even if an answer is selected, because one of the choices is to award an existing answer. In fact it just showed up. I don't know why the link was missing earlier. @Andreas I'm not here for the reputation either, but I do like to show my appreciation in some way when someone actually answers my question, and does it thoroughly. Thanks again. – Garret Wilson Nov 19 '20 at 04:44