Constructor signature is changed if I define case class in spark-shell (scala-shell)

Question

I define case class in spark-shell or scala-shell, and found its constructor signature is changed, there's one extra parameter $iw added, what is that? why that happens? This issue caused me unable to create new instance via its constructor in reflection

scala> case class People(id:Int, name: String)
class People

scala> classOf[People].getDeclaredConstructors.head
val res0: java.lang.reflect.Constructor[_] = public People($iw,int,java.lang.String)

Try to run the repl with -Xprint:typer argument. – Amit Kumar Jan 29 '22 at 11:20 — Amit Kumar, Jan 29 '22 at 11:20
Scala-Shell and Scala-Worksheet should not be trusted. – sarveshseri Feb 09 '22 at 14:08 — sarveshseri, Feb 09 '22 at 14:08

score 2 · Accepted Answer · answered Feb 08 '22 at 18:47

This is expected behavior in both scala-shell and spark-shell after scala 2.12. If you just use spark-shell for quick experiments, you could try spark 2.4 which relies on scala 2.11. For prod, you still can build your package by using relection on case case contructors.

For more details, please refer

snakeyaml and spark results in an inability to construct objects

Spark adds hidden parameter to constructor of a Scala class

Constructor signature is changed if I define case class in spark-shell (scala-shell)

1 Answers1