You can investigate how implicits are resolved now with reify {...} or -Xprint:typer. Then it will be seen what causes ClassCastException.
In scala 2 or 3, is it possible to debug implicit resolution process in runtime?
I haven't investigated yet how implicits are resolved now. Below is just what I think so far.
Firstly, when you define Encoder[Any] you define an encoder literally for everything.
And you define it locally, so this is a high-priority implicit, which fits for everything.
Apparently, this breaks something in Circe machinery. shapeless.labelled.KeyTag starts to be encoded not as expected etc.
Such implicits should be of low priority. Normally, implicits of low priority are defined with LowPriority pattern
trait LowPriority1 {
// implicits
}
trait LowPriority extends LowPriority1 {
// implicits
}
object CompanionObject extents LowPriority {
// implicits
}
But you define it locally. It's tricky to define low-priority local implicits. For example you can use shapeless.LowPriority
implicit def valueEncoderValue(implicit lp: LowPriority): Encoder[Any] ...
Secondly, types Nothing and Any play special role in implicit resolution. Compiler tries to avoid to infer them during implicit resolution. You can check that even AnyRef can improve the situation. Workarounds are to use type T <: Nothing and similarly we can try type T >: Any.
Failed implicit resolution for Nothing with <:<
Thirdly, when you define a type A, normally the best place for implicits Encoder[A], Decoder[A], Codec[A] are in the companion object of A. This can improve automatic/semi-automatic derivation.
So as a workaround try
import io.circe.generic.semiauto.deriveEncoder
import io.circe.Encoder
import shapeless.LowPriority
import io.circe.syntax._
type T >: Any
case class Demo(
field1: T
)
val myDemo = Demo(field1 = None)
print(myDemo.asJson+"\n")
object Demo {
implicit val DemoCodec: Encoder[Demo] = deriveEncoder[Demo]
}
implicit def valueEncoderValue(implicit lp: LowPriority): Encoder[T] = Encoder.encodeString.contramap[T](x=>{
x.toString})
https://scastie.scala-lang.org/DmytroMitin/gaZw3M0zQ6qwNwhqD1nq5g/1
Generally, defining Encoder[Any] looks weird. This looks like kind of abusing type-based codec derivation.
-Xprint:typer shows that deriveCodec[Demo] and myDemo.asJson are resolved in the following way
https://scastie.scala-lang.org/DmytroMitin/gaZw3M0zQ6qwNwhqD1nq5g/3
I was debugging and found the reasons for runtime exceptions. ClassCastException is because shapeless.labelled.KeyTag is a trait rather than an abstract type. Normally this works but not for Any. Not sure whether this is a bug. https://github.com/milessabin/shapeless/issues/1285
type FieldType[K, +V] = V with KeyTag[K, V]
trait KeyTag[K, +V]
trait Tagged[U]
type @@[+T, U] = T with Tagged[U]
None.asInstanceOf[FieldType[Symbol @@ "field1", Any]]
// scala.None$ cannot be cast to KeyTag
type FieldType[K, +V] = V with KeyTag[K, V]
type KeyTag[K, +V]
trait Tagged[U]
type @@[+T, U] = T with Tagged[U]
None.asInstanceOf[FieldType[Symbol @@ "field1", Any]] // no exception
You can try patched versions of Shapeless and Circe-generic with KeyTag being an abstract type rather than trait
scalaVersion := "2.13.10"
resolvers ++= Resolver.sonatypeOssRepos("releases")
libraryDependencies ++= Seq(
"io.circe" %% "circe-core" % "0.14.3",
"io.circe" %% "circe-parser" % "0.14.3",
"com.github.dmytromitin" %% "circe-generic-patched-type-keytag" % "0.14.3",
// "io.circe" %% "circe-generic" % "0.14.3" exclude("com.chuusai", "shapeless_2.13"),
"com.github.dmytromitin" %% "shapeless-patched-type-keytag" % "2.3.10",
// "com.chuusai" %% "shapeless" % "2.3.10",
)
https://github.com/DmytroMitin/shapeless-circe-patched-type-keytag
Then your code throws NullPointerException (ClassCastException is already fixed)
object Main extends App {
case class Demo(
field1: Any
)
val myDemo = Demo(field1 = None)
print(myDemo.asJson + "\n")
implicit val valueEncoderValue: Encoder[Any] = Encoder.encodeString.contramap[Any](x => {
x.toString
})
implicit val valueDecoderValue: Decoder[Any] = Decoder.decodeString.map[Any](x => {
if (x == "Any")
x.asInstanceOf[Any]
else
x.toString
})
implicit lazy val DemoCodec: Codec[Demo] =
deriveCodec[Demo]
}
NullPointerException is because of the initialization order in the object. You can check that
val myDemo = Demo(field1 = None)
implicit val valueEncoderValue: Encoder[Any] = Encoder.encodeString.contramap[Any](_.toString)
implicit val DemoCodec: Codec[Demo] = deriveCodec[Demo]
print(myDemo.asJson + "\n")
//{
// "field1" : "None"
//}
works properly but
val myDemo = Demo(field1 = None)
implicit val valueEncoderValue: Encoder[Any] = Encoder.encodeString.contramap[Any](_.toString)
print(myDemo.asJson + "\n")
implicit val DemoCodec: Codec[Demo] = deriveCodec[Demo]
or
val myDemo = Demo(field1 = None)
implicit val DemoCodec: Codec[Demo] = deriveCodec[Demo]
print(myDemo.asJson + "\n")
implicit val valueEncoderValue: Encoder[Any] = Encoder.encodeString.contramap[Any](_.toString)
throw NullPointerException. The thing is that an implicit (a val) is used when in the object it's not initialized yet. A solution is to use lazy val
print(myDemo.asJson + "\n")
lazy val myDemo = Demo(field1 = None)
implicit lazy val DemoCodec: Codec[Demo] = deriveCodec[Demo]
implicit lazy val valueEncoderValue: Encoder[Any] = Encoder.encodeString.contramap[Any](_.toString)
