Scala @specialized annotation infinite recursion?

Question

Version: scala 2.11.8

I defined a class with specialized type and override method in inheritance:

class Father[@specialized(Int) A]{
  def get(from: A): A = from
}

class Son extends Father[Int]{
  override def get(from: Int): Int = {
    println("Son.get")
    super.get(from)
  }
}

new Son().get(1)  // will cause infinite recursion

So, how to reuse the method of superclass with specialized annotation?

Turning `Father` into a trait seems to solve the problem, but I'm not sure about the reason for that. — adamwy, Dec 26 '16 at 09:30

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

From the article Quirks of Scala Specialization:

Avoid super calls

Qualified super calls are (perhaps fundamentally) broken with specialization. Rewiring the super-accessor methods properly in the specialization phase is a nightmare that has not been solved so far. So, avoid them like the plague, at least for now. In particular, stackable modifications pattern will not work with it well.

So it's most likely a compiler bug, in general you shouldn't use super calls with Scala specialization.

After a bit of investigation:

javap -c Son.class

public class Son extends Father$mcI$sp {
  public int get(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokevirtual #14                 // Method get$mcI$sp:(I)I
       5: ireturn

  public int get$mcI$sp(int);
    Code:
       0: getstatic     #23                 // Field scala/Predef$.MODULE$:Lscala/Predef$;
       3: ldc           #25                 // String Son.get
       5: invokevirtual #29                 // Method scala/Predef$.println:(Ljava/lang/Object;)V
       8: aload_0
       9: iload_1
      10: invokespecial #31                 // Method Father$mcI$sp.get:(I)I
      13: ireturn

Son.get(int) calls Son.get$mcI$sp(int) which turns into Father$mcI$sp.get(int):

javap -c Father\$mcI\$sp.class 

public class Father$mcI$sp extends Father<java.lang.Object> {
  public int get(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokevirtual #12                 // Method get$mcI$sp:(I)I
       5: ireturn

  public int get$mcI$sp(int);
    Code:
       0: iload_1
       1: ireturn

Looks like we have found the cause - Father$mcI$sp.get(int) makes a virtual call to get$mcI$sp, which is overloaded in Son! This is what caused the infinite recursion here.

The compiler has to create specialized versions of method get which is get$mcI$sp, in order to support the non-specialized generic version of Father[T], which unfortunately makes it impossible to have super calls with specialized classes.

Now what happens after changing Father to be a trait (with Scala 2.12):

javap -c Son.class

public class Son implements Father$mcI$sp {

  public int get$mcI$sp(int);
    Code:
       0: getstatic     #25                 // Field scala/Predef$.MODULE$:Lscala/Predef$;
       3: ldc           #27                 // String Son.get
       5: invokevirtual #31                 // Method scala/Predef$.println:(Ljava/lang/Object;)V
       8: aload_0
       9: iload_1
      10: invokestatic  #37                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;
      13: invokestatic  #43                 // InterfaceMethod Father.get$:(LFather;Ljava/lang/Object;)Ljava/lang/Object;
      16: invokestatic  #47                 // Method scala/runtime/BoxesRunTime.unboxToInt:(Ljava/lang/Object;)I
      19: ireturn

It looks like instead of calling get$mcI$sp in a parent class, it invokes static method Father.get$:

javap -c Father.class 

public interface Father<A> {
  public static java.lang.Object get$(Father, java.lang.Object);
    Code:
       0: aload_0
       1: aload_1
       2: invokespecial #17                 // InterfaceMethod get:(Ljava/lang/Object;)Ljava/lang/Object;
       5: areturn

  public A get(A);
    Code:
       0: aload_1
       1: areturn

  public static int get$mcI$sp$(Father, int);
    Code:
       0: aload_0
       1: iload_1
       2: invokespecial #26                 // InterfaceMethod get$mcI$sp:(I)I
       5: ireturn

  public int get$mcI$sp(int);
    Code:
       0: aload_0
       1: iload_1
       2: invokestatic  #33                 // Method scala/runtime/BoxesRunTime.boxToInteger:(I)Ljava/lang/Integer;
       5: invokeinterface #17,  2           // InterfaceMethod get:(Ljava/lang/Object;)Ljava/lang/Object;
      10: invokestatic  #37                 // Method scala/runtime/BoxesRunTime.unboxToInt:(Ljava/lang/Object;)I
      13: ireturn

What's interesting here, is that it seems like the get method is not getting real specialization since it has to box the value in get$mcI$sp, which might be a bug, or maybe specialization support for traits was dropped in Scala 2.12.

It makes it work only with Scala 2.12 (because traits use Java 8 default methods which are handled differently), in Scala 2.11 making it a trait causes infinite recursion too. — adamwy, Dec 26 '16 at 10:33
I see. So this persists unless your trait is SAM converted in Scala 2.12 — Yuval Itzchakov, Dec 26 '16 at 10:43
I think the reason traits fix it is that they use `invokespecial` in the static helpers instead of `invokevirtual`, which bypasses virtual invocation. Theoretically, the issue can be fixed for normal classes too by using `invokespecial` in `Father$mcI$sp::get`. — HTNW, Dec 26 '16 at 16:28

Scala @specialized annotation infinite recursion?

1 Answers1