22

I am looking for an explanation for the following behavior:

  • I have 6 classes, {a.A,b.B,c.C,a.D,b.E,c.F}, each having a package visible m() method that writes out the class name.
  • I have an a.Main class with a main method that does some testing of these classes.
  • The output seems to not follow proper inheritance rules.

Here are the classes:

package a;

public class A {
    void m() { System.out.println("A"); }
}

// ------ 

package b;

import a.A;

public class B extends A {
    void m() { System.out.println("B"); }
}

// ------ 

package c;

import b.B;

public class C extends B {
    void m() { System.out.println("C"); }
}

// ------ 

package a;

import c.C;

public class D extends C {
    void m() { System.out.println("D"); }
}

// ------ 

package b;

import a.D;

public class E extends D {
    void m() { System.out.println("E"); }
}

// ------ 

package c;

import b.E;

public class F extends E {
    void m() { System.out.println("F"); }
}

The Main class is in package a:

package a;

import b.B;
import b.E;
import c.C;
import c.F;

public class Main {

    public static void main(String[] args) {
        A a = new A();
        B b = new B();
        C c = new C();
        D d = new D();
        E e = new E();
        F f = new F();

        System.out.println("((A)a).m();"); ((A)a).m();
        System.out.println("((A)b).m();"); ((A)b).m();
        System.out.println("((A)c).m();"); ((A)c).m();
        System.out.println("((A)d).m();"); ((A)d).m();
        System.out.println("((A)e).m();"); ((A)e).m();
        System.out.println("((A)f).m();"); ((A)f).m();

        System.out.println("((D)d).m();"); ((D)d).m();
        System.out.println("((D)e).m();"); ((D)e).m();
        System.out.println("((D)f).m();"); ((D)f).m();
    }
}

And here is the output:

((A)a).m();
A
((A)b).m();
A
((A)c).m();
A
((A)d).m();
D
((A)e).m();
E
((A)f).m();
F
((D)d).m();
D
((D)e).m();
D
((D)f).m();
D

And here are my questions:

1) I understand that D.m() hides A.m(), but a cast to A should expose the hidden m() method, is that true? Or is D.m() overrides A.m() in spite of the fact that B.m() and C.m() breaks the inheritance chain?

((A)d).m();
D

2) Even worse, the following code shows overriding in effect, why? 

((A)e).m();
E
((A)f).m();
F

And why not in this part:

((A)a).m();
A
((A)b).m();
A
((A)c).m();
A

and this one?

((D)d).m();
D
((D)e).m();
D
((D)f).m();
D

I am using OpenJDK javac 11.0.2.

EDIT: The first question is answered by How to override a method with default (package) visibility scope?

An instance method mD declared in or inherited by class D, overrides from D another method mA declared in class A, iff all of the following are true:

  • A is a superclass of D.
  • D does not inherit mA (because crossing package boundaries)
  • The signature of mD is a subsignature (§8.4.2) of the signature of mA.
  • One of the following is true: [...]
    • mA is declared with package access in the same package as D (this case), and either D declares mD or mA is a member of the direct superclass of D. [...]

BUT: the second question is still unresolved.

Right leg
  • 16,080
  • 7
  • 48
  • 81
TFuto
  • 1,361
  • 15
  • 33
  • 12
    I liked the question, but, dude, did you really need *six* classes to make the point? – Andrew Tobilko Sep 22 '19 at 19:24
  • 1
    @Andrew Tobilko: Valid point, sorry. I guess then you wouldn't have been happy with the initial write-up with 12 classes. :-) – TFuto Sep 22 '19 at 19:28
  • @dyukha: I fail to yet see how the suggested duplicate link explains the virtual dispatch... But maybe it is just me. – TFuto Sep 22 '19 at 19:36
  • The answer to your first question is: false. Casting does not change which method is invoked at runtime. If you cast a `List` to an Object and call its toString method, you won’t be calling the default toString method of Object. Your second question exposes a Java bug, in my opinion. It should never be possible to invoke package-private functionality in another package. – VGR Sep 22 '19 at 20:48
  • 1
    Apparently the confusion started 20 years ago... See [link1](https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4153303), [link2](https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4089922) and [link3](https://bugs.java.com/bugdatabase/view_bug.do?bug_id=1240831) from the bug database. E.g. if a class is only abstract because of an abstract package-visible method, then is a class in another package, derived from the former class, abstract? Etc... – TFuto Sep 23 '19 at 07:36
  • I got a confirmation that this is a [bug](https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8232678). – TFuto Oct 21 '19 at 12:05

4 Answers4

7

I understand that D.m() hides A.m(), but a cast to A should expose the hidden m() method, is that true?

There is no such thing as hiding for instance (non-static) methods. Here, it's an example of shadowing. A cast to A in most places just helps to resolve the ambiguity (e.g. c.m() as is can refer to both A#m and C#m [which isn't accessible from a]) that otherwise would lead to a compilation error.

Or is D.m() overrides A.m() in spite of the fact that B.m() and C.m() breaks the inheritance chain?

b.m() is an ambiguous call because both A#m and B#m are applicable if you set the visibility factor aside. The same goes for c.m(). ((A)b).m() and ((A)c).m() clearly refer to A#m which is accessible for the caller.

((A)d).m() is more interesting: both A and D reside in the same package (thus, accessible [which is different from the two above cases]) and D indirectly inherits A. During dynamic dispatch, Java will be able to call D#m because D#m actually overrides A#m and there is no reason not to call it (despite the mess going on the inheritance path [remember that neither B#m nor C#m overrides A#m due to the visibility issue]).

Even worse, the following code shows overriding in effect, why?

I can't explain this because it's not the behaviour I expected.

I dare say that the result of 

((A)e).m();
((A)f).m();

should be identical to the result of

((D)e).m();
((D)f).m();

which is 

D
D

since there is no way to access the package-private methods in b and c from a.

Andrew Tobilko
  • 48,120
  • 14
  • 91
  • 142
3

Interesting question. I checked that in Oracle JDK 13 and Open JDK 13. Both give the same result, exactly as you wrote. But this result contradicts with Java Language Specification.

Unlike class D, which is in the same package as A, classes B, C, E, F are in a different package and because of package private declaration of A.m() cannot see it and cannot override it. For classes B and C it works as specified in JLS. But for classes E and F it doesn't. The cases with ((A)e).m() and ((A)f).m() are bugs in the implementation of Java compiler.

How should work ((A)e).m() and ((A)f).m()? Since D.m() overrides A.m(), this should hold also for all their subclasses. Thus both ((A)e).m() and ((A)f).m() should be the same as ((D)e).m() and ((D)f).m(), means they all should call D.m().

mentallurg
  • 4,967
  • 5
  • 28
  • 36
  • I find nothing wrong with both `((A)e).m()` and `((A)f).m()` - they should compile. The result they produce is weird, though. – Andrew Tobilko Sep 22 '19 at 21:25
  • They do compile. But yes, not matter to what class in the hierarchy you cast, if a method is visible, the execution result must be the same. – mentallurg Sep 22 '19 at 21:30
3

This is a brain teaser, indeed.

The following answer is not yet fully conclusive, but my results of having a short look at this. Maybe it at least contributes to finding a definite answer. Parts of the question have already been answered, so I'm focussing on the point that still causes confusion and is not explained yet.

The critical case can be boiled down to four classes:

package a;

public class A {
    void m() { System.out.println("A"); }
}

package a;

import b.B;

public class D extends B {
    @Override
    void m() { System.out.println("D"); }
}

package b;

import a.A;

public class B extends A {
    void m() { System.out.println("B"); }
}

package b;

import a.D;

public class E extends D {
    @Override
    void m() { System.out.println("E"); }
}

(Note that I added @Override annotations where possible - I hoped that this could already give a hint, but I wasnt' able to draw conclusions from that yet...)

And the main class:

package a;

import b.E;

public class Main {

    public static void main(String[] args) {

        D d = new D();
        E e = new E();
        System.out.print("((A)d).m();"); ((A) d).m();
        System.out.print("((A)e).m();"); ((A) e).m();

        System.out.print("((D)d).m();"); ((D) d).m();
        System.out.print("((D)e).m();"); ((D) e).m();
    }

}

The unexpected output here is

((A)d).m();D
((A)e).m();E
((D)d).m();D
((D)e).m();D

So

  • when casting an object of type D to A, the method from type D is called
  • when casting an object of type E to A, the method from type E is called (!)
  • when casting an object of type D to D, the method from type D is called
  • when casting an object of type E to D, the method from type D is called

It's easy to spot the odd one out here: One would naturally expect that casting an E to A should cause the method of D to be called, because that's the "highest" method in the same package. The observed behavior cannot easily be explained from the JLS, although one would have to re-read it, carefully, to be sure that there's not a subtle reason for that.

Out of curiosity, I had a look at the generated bytecode of the Main class. This is the entire output of javap -c -v Main (the relevant parts will be fleshed out below):

public class a.Main
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC, ACC_SUPER
Constant pool:
   #1 = Class              #2             // a/Main
   #2 = Utf8               a/Main
   #3 = Class              #4             // java/lang/Object
   #4 = Utf8               java/lang/Object
   #5 = Utf8               <init>
   #6 = Utf8               ()V
   #7 = Utf8               Code
   #8 = Methodref          #3.#9          // java/lang/Object."<init>":()V
   #9 = NameAndType        #5:#6          // "<init>":()V
  #10 = Utf8               LineNumberTable
  #11 = Utf8               LocalVariableTable
  #12 = Utf8               this
  #13 = Utf8               La/Main;
  #14 = Utf8               main
  #15 = Utf8               ([Ljava/lang/String;)V
  #16 = Class              #17            // a/D
  #17 = Utf8               a/D
  #18 = Methodref          #16.#9         // a/D."<init>":()V
  #19 = Class              #20            // b/E
  #20 = Utf8               b/E
  #21 = Methodref          #19.#9         // b/E."<init>":()V
  #22 = Fieldref           #23.#25        // java/lang/System.out:Ljava/io/PrintStream;
  #23 = Class              #24            // java/lang/System
  #24 = Utf8               java/lang/System
  #25 = NameAndType        #26:#27        // out:Ljava/io/PrintStream;
  #26 = Utf8               out
  #27 = Utf8               Ljava/io/PrintStream;
  #28 = String             #29            // ((A)d).m();
  #29 = Utf8               ((A)d).m();
  #30 = Methodref          #31.#33        // java/io/PrintStream.print:(Ljava/lang/String;)V
  #31 = Class              #32            // java/io/PrintStream
  #32 = Utf8               java/io/PrintStream
  #33 = NameAndType        #34:#35        // print:(Ljava/lang/String;)V
  #34 = Utf8               print
  #35 = Utf8               (Ljava/lang/String;)V
  #36 = Methodref          #37.#39        // a/A.m:()V
  #37 = Class              #38            // a/A
  #38 = Utf8               a/A
  #39 = NameAndType        #40:#6         // m:()V
  #40 = Utf8               m
  #41 = String             #42            // ((A)e).m();
  #42 = Utf8               ((A)e).m();
  #43 = String             #44            // ((D)d).m();
  #44 = Utf8               ((D)d).m();
  #45 = Methodref          #16.#39        // a/D.m:()V
  #46 = String             #47            // ((D)e).m();
  #47 = Utf8               ((D)e).m();
  #48 = Utf8               args
  #49 = Utf8               [Ljava/lang/String;
  #50 = Utf8               d
  #51 = Utf8               La/D;
  #52 = Utf8               e
  #53 = Utf8               Lb/E;
  #54 = Utf8               SourceFile
  #55 = Utf8               Main.java
{
  public a.Main();
    descriptor: ()V
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokespecial #8                  // Method java/lang/Object."<init>":()V
         4: return
      LineNumberTable:
        line 5: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   La/Main;

  public static void main(java.lang.String[]);
    descriptor: ([Ljava/lang/String;)V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=3, args_size=1
         0: new           #16                 // class a/D
         3: dup
         4: invokespecial #18                 // Method a/D."<init>":()V
         7: astore_1
         8: new           #19                 // class b/E
        11: dup
        12: invokespecial #21                 // Method b/E."<init>":()V
        15: astore_2
        16: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
        19: ldc           #28                 // String ((A)d).m();
        21: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
        24: aload_1
        25: invokevirtual #36                 // Method a/A.m:()V
        28: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
        31: ldc           #41                 // String ((A)e).m();
        33: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
        36: aload_2
        37: invokevirtual #36                 // Method a/A.m:()V
        40: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
        43: ldc           #43                 // String ((D)d).m();
        45: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
        48: aload_1
        49: invokevirtual #45                 // Method a/D.m:()V
        52: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
        55: ldc           #46                 // String ((D)e).m();
        57: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
        60: aload_2
        61: invokevirtual #45                 // Method a/D.m:()V
        64: return
      LineNumberTable:
        line 9: 0
        line 10: 8
        line 11: 16
        line 12: 28
        line 14: 40
        line 15: 52
        line 16: 64
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0      65     0  args   [Ljava/lang/String;
            8      57     1     d   La/D;
           16      49     2     e   Lb/E;
}
SourceFile: "Main.java"

The interesting thing is the invocation of the methods:

16: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
19: ldc           #28                 // String ((A)d).m();
21: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
24: aload_1
25: invokevirtual #36                 // Method a/A.m:()V

28: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
31: ldc           #41                 // String ((A)e).m();
33: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
36: aload_2
37: invokevirtual #36                 // Method a/A.m:()V

40: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
43: ldc           #43                 // String ((D)d).m();
45: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
48: aload_1
49: invokevirtual #45                 // Method a/D.m:()V

52: getstatic     #22                 // Field java/lang/System.out:Ljava/io/PrintStream;
55: ldc           #46                 // String ((D)e).m();
57: invokevirtual #30                 // Method java/io/PrintStream.print:(Ljava/lang/String;)V
60: aload_2
61: invokevirtual #45                 // Method a/D.m:()V

The bytecode explicitly refers to the method A.m in the first two calls, and explicitly refers to the method D.m in the second calls.

One conclusion that I draw from that: The culprit is not the compiler, but the handling of the invokevirtual instruction of the JVM!

The documentation of invokevirtual does not contain any surprises - quoting only the relevant part here:

Let C be the class of objectref. The actual method to be invoked is selected by the following lookup procedure:

  1. If C contains a declaration for an instance method m that overrides (§5.4.5) the resolved method, then m is the method to be invoked.

  2. Otherwise, if C has a superclass, a search for a declaration of an instance method that overrides the resolved method is performed, starting with the direct superclass of C and continuing with the direct superclass of that class, and so forth, until an overriding method is found or no further superclasses exist. If an overriding method is found, it is the method to be invoked.

  3. Otherwise, if there is exactly one maximally-specific method (§5.4.3.3) in the superinterfaces of C that matches the resolved method's name and descriptor and is not abstract, then it is the method to be invoked.

It supposedly just goes up the hierarchy, until it finds a method that (is or) overrides the method, with overrides (§5.4.5) being defined as one would naturally expect.

Still no obvious reason for the observed behavior.

I then started looking at what actually happens when an invokevirtual is encountered, and drilled down into the LinkResolver::resolve_method function of the OpenJDK, but at that point, I'm not entirely sure whether this is the right place to look at, and I currently cannot invest more time here...

Maybe others can continue from here, or find inspiration for their own investigations. At least the fact that the compiler does the right thing, and the quirk seems to be in the handling of invokevirtual, might be a starting point.

Marco13
  • 53,703
  • 9
  • 80
  • 159
3

I reported this issue and it was confirmed a bug for several Java versions.

Bug report.

I mark this answer as the solution, but would like to thank everyone for all the answers and messages, I learned a lot. :-)

TFuto
  • 1,361
  • 15
  • 33