3

The following snippet throws NullPointerException. Is it expected and normal behavior of Scala?

object ATest extends App {
    def getX[T <: X](constr: ⇒ T = null.asInstanceOf[T]): Unit = {
        constr
    }
    getX()
}
class X

Generated (decompied) Java code from snippet:

public final class ATest {
public static void main(String[] arrstring) {
    ATest$.MODULE$.main(arrstring);
}
public static void delayedInit(Function0<BoxedUnit> function0) {
    ATest$.MODULE$.delayedInit(function0);
}
public static String[] args() {
    return ATest$.MODULE$.args();
}
public static void scala$App$_setter_$executionStart_$eq(long l) {
    ATest$.MODULE$.scala$App$_setter_$executionStart_$eq(l);
}
public static long executionStart() {
    return ATest$.MODULE$.executionStart();
}
public static void delayedEndpoint$test$ATest$1() {
    ATest$.MODULE$.delayedEndpoint$test$ATest$1();
}
public static <T extends X> T getX$default$1() {
    return ATest$.MODULE$.getX$default$1();
}
public static <T extends X> void getX(Function0<T> function0) {
    ATest$.MODULE$.getX(function0);
}
}


public final class ATest$ implements App {
public static final ATest$ MODULE$;
private final long executionStart;
private String[] scala$App$$_args;
private final ListBuffer<Function0<BoxedUnit>> scala$App$$initCode;

public static {
    new test.ATest$();
}
public long executionStart() {
    return this.executionStart;
}
public String[] scala$App$$_args() {
    return this.scala$App$$_args;
}
public void scala$App$$_args_$eq(String[] x$1) {
    this.scala$App$$_args = x$1;
}
public ListBuffer<Function0<BoxedUnit>> scala$App$$initCode() {
    return this.scala$App$$initCode;
}
public void scala$App$_setter_$executionStart_$eq(long x$1) {
    this.executionStart = x$1;
}
public void scala$App$_setter_$scala$App$$initCode_$eq(ListBuffer x$1) {
    this.scala$App$$initCode = x$1;
}
public String[] args() {
    return App.class.args((App)this);
}
public void delayedInit(Function0<BoxedUnit> body) {
    App.class.delayedInit((App)this, body);
}
public void main(String[] args) {
    App.class.main((App)this, (String[])args);
}
public <T extends X> void getX(Function0<T> constr) {
    constr.apply();
}
public <T extends X> T getX$default$1() {
    return null;
}
public final void delayedEndpoint$test$ATest$1() {
    this.getX((Function0<T>)new scala.Serializable(){
        public static final long serialVersionUID = 0;

        public final Nothing. apply() {
            return (Nothing.)ATest$.MODULE$.getX$default$1();
        }
    });
}
private ATest$() {
    MODULE$ = this;
    App.class.$init$((App)this);
    this.delayedInit((Function0<BoxedUnit>)new ATest.delayedInit$body(this));
}
}

public final class ATest$.anonfun extends AbstractFunction0<Nothing.>implements Serializable {
public final Nothing. apply() {
        return (Nothing.)ATest$.MODULE$.getX$default$1();
        }
}

And finally action part:

public <T extends X> void getX(Function0<T> constr) {
    constr.apply();
}
public <T extends X> T getX$default$1() {
    return null;
}
public final void delayedEndpoint$test$ATest$1() {
    this.getX((Function0<T>)new scala.Serializable(){
        public final Nothing. apply() {
            return (Nothing.)ATest$.MODULE$.getX$default$1();
        }
    });
}

That is: call to getX passes new anon Function0 which apply() just calls getX$default$1() that is null. So i can not see any point where NPE can be thrown.

EDIT: The unresolved issue is found: https://issues.scala-lang.org/browse/SI-8097

EDIT: Expression null.asInstanceOf[T] generates default value for type T. In case when Scala infers resulting type T as Nothing we come to runtime expression

null.asInstanceOf[Nothing]

that obviously throws Exeption as a default for Nothing is Exception.

But than why this snippet throws NPE only at last line?

object ATest extends App {
    def getX[T](x: T = null.asInstanceOf[T]): T = x
    getX[Nothing]() // Ok
    val x = getX() // Ok 
    val y = null
    println("x= "+x) // prints 'x= null'
    println(s"y= $y") // prints 'y= null'
    println(s"x= $x") // throws NPE !?
    println("x==null ? "+(x==null)) // prints 'x= null'
}

And why this snippet throws NPE (it is only different from previous in implicit param)?

object ATest extends App {
    def getX[T](x: T = null.asInstanceOf[T])(implicit s: String = null): T = x
    getX() // throws NPE !?
}

So the situation is still vague. And question is open.

Markus Marvell
  • 1,916
  • 1
  • 21
  • 26
  • I cannot reproduce the NPE with Scala 2.11.7, see https://gist.github.com/lrytz/1c05e0fa666b130e2d38 - what version did you use? – Lukas Rytz Feb 06 '16 at 21:45
  • I *can* reproduce this with Scala 2.11.7 and Java 1.7. – Kolmar Feb 06 '16 at 21:49
  • Aw, my bad, ran the wrong main. – Lukas Rytz Feb 06 '16 at 22:15
  • 1
    The example does not NPE when using Scala 2.12.0-M3 - unless I got it wrong again, confirmation welcome! I think the crash is actually correct (a different exception may be more useful), and 2.12.0-M3 is probably buggy. The issue is that type checker infers Nothing for the type parameter T (use -Xprint:typer), and null is not a valid value for type Nothing (there are no valid types for Nothing). So null.asInstanceOf[Nothing] crashes - what else should it do? – Lukas Rytz Feb 06 '16 at 22:27
  • Interessting, I can confirm that. Also in repl if you try "".instanceOf[Nothing] you do not get a null pointer anymore, instead, you get told, that Nothing can not be used in instanceOf. It looks like they got rid of that code but aren't handling that case correctly still. – dth Feb 06 '16 at 23:59
  • If I do this in repl `def foo[T](x: Any) = x.asInstanceOf[T]` I probably should get a compiler error as T has not a lower bound that excludes nothing, but instead it compiles. When I then do this `foo("asd")` i get: java.lang.ClassCastException: java.lang.String cannot be cast to scala.runtime.Nothing$ which is more informative but still not good. And I should get the same error in the example here, but somehow this is optimized away somewhere – dth Feb 07 '16 at 00:04
  • there's a number of relevant tickets: https://issues.scala-lang.org/browse/SI-4264?jql=text%20~%20%22nullpointerexception%20nothing%22 – Lukas Rytz Feb 07 '16 at 09:27
  • @Lukas Rytz Seems you are correct. In case when Scala infers resulting type as Nothing it throws NPE that is quite expected. I.e. an expression null.asInstanceOf[Nothing] throws NPE. So the answer to the thread question is: It's expected behavior and not a bug. May be an exception should be more informative. – Markus Marvell Feb 09 '16 at 00:14

1 Answers1

4

So I have to revise my answer a bit.

What triggers the NPE is however clear from the byte code, but not the reverse compiled Java code. Byte code has more features than Java code, the important one being, that you can have two methods that differ only in the return type and do different things.

So lets first look at the stack trace:

at ATest$$anonfun$1.apply(Test.scala:7)
at ATest$.getX(Test.scala:5)
at ATest$.delayedEndpoint$ATest$1(Test.scala:7)
at ATest$delayedInit$body.apply(Test.scala:3)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:383)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at ATest$.main(Test.scala:3)
at ATest.main(Test.scala)

So the method where it all goes wrong is ATest$$anonfun$1.apply

Lets look at that:

public final scala.runtime.Nothing$ apply();
Code:
   0: getstatic     #19                 // Field ATest$.MODULE$:LATest$;
   3: invokevirtual #23                 // Method ATest$.getX$default$1:()LX;
   6: checkcast     #25                 // class scala/runtime/Nothing$
   9: areturn

public final java.lang.Object apply();
Code:
   0: aload_0
   1: invokevirtual #30                 // Method apply:()Lscala/runtime/Nothing$;
   4: athrow

The first thing we notice is there are two Methods called apply, so which one is called (the athrow is a hint...) Well, lets look at the method calling it:

public <T extends X> void getX(scala.Function0<T>);
Code:
   0: aload_1
   1: invokeinterface #62,  1  // InterfaceMethod scala/Function0.apply:()Ljava/lang/Object;
   6: pop
   7: return

So, we are calling the one that returns an Object and has the athrow instruction. So why does that give us a NullPointer exception?

Well, the method does the following: it places this on the stack, then it invokes the other apply method (returning a Nothing$), this method actually returns null as it returns our default argument. And now we have a null on the stack and execute athrow. And athrow throws a NPE instead if it finds a null on the stack.

So this is what happens here.

The next question is, why does it happen?

Well, lets look at what scalac makes of it after typechecking:

object ATest extends AnyRef with App {
    def <init>(): ATest.type = {
      ATest.super.<init>();
      ()
    };
    def getX[T <: X](constr: => T = null.asInstanceOf[T]): Unit = {
      constr;
      ()
    };
    <synthetic> def getX$default$1[T <: X]: T = null.asInstanceOf[T];
    ATest.this.getX[Nothing](ATest.this.getX$default$1[Nothing])
  }

And what it does in the case without the asInstanceOf:

object ATest extends AnyRef with App {
    def <init>(): ATest.type = {
      ATest.super.<init>();
      ()
    };
    def getX[T <: X](constr: => T = null): Unit = {
      constr;
      ()
    };
    <synthetic> def getX$default$1[T <: X]: Null = null;
    ATest.this.getX[Null](ATest.this.getX$default$1[Nothing])
  }

Well, somehow the information, that the default parameter is Null is lost in the first case.

In the second case we get this byte code for the critical method:

public final java.lang.Object apply();
Code:
   0: aload_0
   1: invokevirtual #27                 // Method apply:()Lscala/runtime/Null$;
   4: pop
   5: aconst_null
   6: areturn

So here, the compiler knows, that the argument is null, and generates code to box null using the class Null$.

What should happen?

Well, not a null pointer exception, for sure. But why does the compiler generate that athrow in the first place? Probably because of the asInstanceOf[T] which becomes an asInstanceOf[Nothing] which should throw an exception if invoked on a null.

Let's quickly try, what happens if we do that in repl:

"".asInstanceOf[Nothing]
java.lang.ClassCastException: java.lang.String cannot be cast to scala.runtime.Nothing$

So far so good, and this:

null.asInstanceOf[Nothing]
java.lang.NullPointerException

Well, maybe I should have started with this... it seems, that code generation for asInstanceOf has some bug and throws the wrong exception.

Why a lower bound :> Null fixes the problem, is also clear: the inferred type is no longer Nothing but Null and the instanceOf is fine.

So the more interesting problem is why the type checker fails on your complex example that you have removed now.

Complex example

class X
object ATest extends App {
  def getX[T<:X](clas: Class[T], constr: ⇒ T = null): T ={
    val x = constr
    if (x == null) clas.newInstance() else x
  }
  val clas: Class[_ <: X] = classOf[X]
  getX(clas) // Ooops: type mismatch..
}

Well, what does the type checker say:

def getX[T <: X](clas: Class[T], constr: => T = null): T = {
  val x: T = constr;
    if (x.==(null))
      clas.newInstance()
    else
      x
  };
  <synthetic> def getX$default$2[T <: X]: Null = null;
  private[this] val clas: Class[_ <: X] = classOf[X];
  <stable> <accessor> def clas: Class[_ <: X] = ATest.this.clas;
  ATest.this.getX[T](<clas: error>, ATest.this.getX$default$2)
}

Somehow he cant infer a type for T, but he should infer Null, because there are only classes for reference types. Interestingly, the compiler does not know that. It seems to directly use the definition from Class from Java and the type parameter there has no lower bound (because Java has no type Null), so the lower bound is Nothing. This also tells us, how to fix it:

val clas: Class[_ >: Null <: X] = classOf[X]
getX(clas)

This finally works. So you can do exactly, what you wanted to do in the first place. You just have to tell the compiler, that you are not interested in classes for types that cannot be nulled.

I think I still prefer the version with Option though:

def getX[T <: X](clas: Class[T], constr: ⇒ Option[T] = None): T = {
  val x = constr
   x match {
    case None => clas.newInstance()
    case Some(x) => x
  }
}
val clas: Class[_ <: X] = classOf[X]
getX(clas)

Its now also clear, why this works: None is a Option[Nothing], so this code can handle a Class[Nothing] just fine.

dth
  • 2,287
  • 10
  • 17
  • i added decompiled code. And still i see no place to throw NPE. – Markus Marvell Feb 06 '16 at 21:20
  • Btw, another way to fix this is to add the usual constraint for abstract types with `null` values: `def getX[T >: Null <: X]` – Kolmar Feb 06 '16 at 21:52
  • so, i hope i have figured out everything now ;) – dth Feb 06 '16 at 23:51
  • Thanks for deep diving into problem. It was interesting. Finally i found an unresolved issue here: https://issues.scala-lang.org/browse/SI-8097 – Markus Marvell Feb 07 '16 at 05:09
  • Not sure if you're aware that `null.asInstanceOf` is in the spec. http://www.scala-lang.org/files/archive/spec/2.11/06-expressions.html#the-null-value The explanation above is not very precise about null and Nothing. – som-snytt Feb 23 '16 at 23:07