How to extract all methods in scala 3 using a tasty inspector?

Question

I am trying to use a tasty inspector to convert method params to case classes but I get a classcast exception at runtime.

My code:

import dotty.tools.dotc.ast.Trees.{PackageDef, Template}

import scala.quoted.*
import scala.tasty.inspector.*

class MyInspector extends Inspector:
  def inspect(using Quotes)(tastys: List[Tasty[quotes.type]]): Unit =
    for tasty <- tastys do
      import tasty.quotes.reflect.*
      tasty.ast match {
        case PackageDef(pid, stats) =>
          stats.collect { case TypeDef(typeName, Template(constr, parentsOrDerived, self, preBody: List[_])) =>
            preBody.collect { case DefDef(name, paramss: List[List[_]] @unchecked, tpt, preRhs) =>
              val params = paramss.flatten.map { case ValDef(name, tpt, preRhs) =>
                s"$name : ${tpt.show}"
              }
              println(s"""
                   |case class ${typeName}_${name}_ccIn(${params.mkString(", ")})
                   |""".stripMargin)
              println("------------------------")
            }
          }
      }

@main def tryit() =
  val tastyFiles = List("../example-commands/classpath-1/target/scala-3.2.1/classes/cp1/Cp1Exports.tasty")
  TastyInspector.inspectTastyFiles(tastyFiles)(new MyInspector)

I run this against this class (after I compile it and the compiler creates a .tasty file):

package cp1

import java.time.LocalDate

trait Cp1Exports:
  def add(a: Int, b: Int): Int
  def subtract(a: Int, b: Int): Int
  def friends(p: Person, from: LocalDate): Seq[Person]

case class Person(id: Int, name: String)

But I get this exception:

Exception in thread "main" java.lang.ClassCastException: class dotty.tools.dotc.ast.Trees$Import cannot be cast to class dotty.tools.dotc.ast.Trees$TypeDef (dotty.tools.dotc.ast.Trees$Import and dotty.tools.dotc.ast.Trees$TypeDef are in unnamed module of loader 'app')
    at scala.quoted.runtime.impl.QuotesImpl$reflect$TypeDef$.unapply(QuotesImpl.scala:339)
    at console.macros.MyInspector$$anon$1.applyOrElse(MyInspector.scala:15)

The line causing the issue is this:

          stats.collect { case TypeDef(typeName, Template(constr, parentsOrDerived, self, preBody: List[_])) =>

But it shouldn't because this is a collect. The error is caused because there is an import in Cp1Exports. If I remove the import, it works.

Also any advice to simplify the code would be appreciated.

I am using scala 3.2.1 (incl scala-compiler with that version)

EDIT:

Ok after following the advice from below, I ended up with this code which works (but seems rather complicated):

import dotty.tools.dotc.ast.Trees.*

import scala.quoted.*
import scala.tasty.inspector.*

class MyInspector extends Inspector:
  def inspect(using Quotes)(tastys: List[Tasty[quotes.type]]): Unit =
    for tasty <- tastys do
      given dotty.tools.dotc.core.Contexts.Context = scala.quoted.quotes.asInstanceOf[scala.quoted.runtime.impl.QuotesImpl].ctx
      tasty.ast match {
        case PackageDef(pid, stats) =>
          stats.collect { case TypeDef(typeName, Template(constr, parentsOrDerived, self, preBody: List[_])) =>
            preBody.collect { case DefDef(name, paramss: List[List[_]] @unchecked, tpt, preRhs) =>
              val params = paramss.flatten.map { case ValDef(name, tpt, preRhs) =>
                s"$name : ${tpt.show}"
              }
              println(s"""
                   |case class ${typeName}_${name}_ccIn(${params.mkString(", ")})
                   |""".stripMargin)
              println("------------------------")
            }
          }
      }

@main def tryit() =
  val tastyFiles = List("../example-commands/classpath-1/target/scala-3.2.1/classes/cp1/Cp1Exports.tasty")
  TastyInspector.inspectTastyFiles(tastyFiles)(new MyInspector)

Thanks

Answer 1

I noticed that

stats.collect { case dotty.tools.dotc.ast.Trees.TypeDef(_, _) => } 

doesn't throw while

stats.collect { TypeDef(_, _) => } 

aka

stats.collect { tasty.quotes.reflect.TypeDef(_, _) => }

does.

The difference seems to be that dotty.tools.dotc.ast.Trees.TypeDef is a case class while tasty.quotes.reflect.TypeDef is an abstract type.

The reason seems to be type erasure

Understanding why inlining function fixes runtime crash when collecting

Simpler reproduction:

import scala.reflect.TypeTest

trait X {
  type A

  type B <: A
  trait BModule {
    def unapply(b: B): Option[Int]
  }
  val B: BModule
  given BTypeTest: TypeTest[A, B]

  type C <: A
  trait CModule {
    def unapply(c: C): Option[String]
  }
  val C: CModule
  given CTypeTest: TypeTest[A, C]
}

object XImpl extends X {
  sealed trait A

  case class B(i: Int) extends A
  object B extends BModule {
    def unapply(b: B): Option[Int] = Some(b.i)
  }
  object BTypeTest extends TypeTest[A, B] {
    override def unapply(x: A): Option[x.type & B] = x match {
      case x: (B & x.type) => Some(x)
      case _ => None
    }
  }

  case class C(s: String) extends A
  object C extends CModule {
    def unapply(c: C): Option[String] = Some(c.s)
  }
  object CTypeTest extends TypeTest[A, C] {
    override def unapply(x: A): Option[x.type & C] = x match {
      case x: (C & x.type) => Some(x)
      case _ => None
    }
  }
}

def foo()(using x: X) = {
  import x.*

  List(XImpl.B(1), XImpl.C("a")).collect { case C(s) => println(s) }
}

given X = XImpl

foo() // ClassCastException: XImpl$B cannot be cast to XImpl$C

Here X, A, B, C are similar to Quotes, Tree, ValDef, TypeDef.

Answer 2

Casting and inspecting runtime types of Quotes is over-complicating. I understand that it comes from poor IDE support, so I'll explain how to avoid it.

Then you import you project into IntelliJ, it will not handle path-dependent types well, and macros - both in Scala 2 and in Scala 3 - rely on them, so the IDE support is poor. However, in Scala 3 you can learn how to work around it.

You are starting with this:

import scala.quoted.*
import scala.tasty.inspector.*

These imports will allow you to create the Inspector instance:

class PrintInspector extends Inspector {

  override def inspect(using quotes: Quotes)(tastys: List[Tasty[quotes.type]]): Unit = ???
}

Pay attention to the fact that Quotes using is named and we are referring to it in second argument list. It allow us to express that for each tasty: Tasty[quotes.type] value:

• tasty.ast will be of type quotes.reflect.Tree
• tasty.quote is the same type as quote (so we might import the latter everywhere)

To work with Quotes and reflect you should do 2 wildcard imports:

// same type as content of each Tasty[quotes.type].quotes
import quotes.*
import quotes.reflect.*

Here IDE support will start getting poor:

• when you open Quotes in your IDE you will see all the companion objects and extension methods that you should be able to access
• however IntelliJ (in my case, at the moment I am writing this) only recognizes imported types, while fails to recognize extension methods and companion object methods
• so I am helping myself annotating types rather than relying on what is inferred to have better IDE experience

How do I work with it? An example:

  tastys.foreach { tasty =>
    val path = tasty.path
    val tree: Tree = tasty.ast
    val symbol: Symbol = tree.symbol
  }

I saw in Tasty's source code that I should have access to tasty.ast method and that is should be Tree (imported from quotes.reflect.*). IntelliJ will assume it's Any so I will annotate it myself.

Similarly I can see in Quotes source code that Tree has .symbol extension method.

This way I am avoiding the madness in which I am casting things to internal compiler types and figuring our implicits to pass that were never intended for API user to pass manually. (And opening Quotes.tasty, Inspector.tasty, Tasty.tasty etc to preview methods can be done in IntelliJ with double shift, typing the name and then double shift again to search in dependencies. Once opened I keep e.g. Quotes.tasty on the right side of the screen and a file which use the API on the left side).

Once we are working with sane-but-not-intellisense supported API we can write this:

//> using scala "3.2.1"
//> using lib "org.scala-lang::scala3-tasty-inspector:3.2.1"

import scala.quoted.*
import scala.tasty.inspector.*

class PrintInspector extends Inspector {

  override def inspect(using quotes: Quotes)(tastys: List[Tasty[quotes.type]]): Unit = {
    // same type as content of each Tasty[quotes.type].quotes
    import quotes.*
    import quotes.reflect.*

    def processSymbol(sym: Symbol): Vector[String] = {
      if (sym == Symbol.noSymbol) {
        Vector.empty
      } else if (sym.isPackageDef) {
        // it's a package
        val name: String = s"package ${sym.name}"
        val decl: List[Symbol] = sym.declarations
        name +: decl.view.flatMap(processSymbol).map("  " + _).toVector
      } else if (sym.isClassDef) {
        // it's a class
        val name: String = s"class ${sym.name}"
        val decl: List[Symbol] = sym.declarations
        name +: decl.view.flatMap(processSymbol).map("  " + _).toVector
      } else if (sym.isDefDef) {
        // it's a def (not val)
        val name: String = s"def ${sym.name}"
        val decl: List[Symbol] = sym.paramSymss.flatten // parameter lists of this method
        name +: decl.view.flatMap(processSymbol).map("  " + _).toVector
      } else if (sym.isValDef) {
        // it's a val
        val name: String = s"val ${sym.name}"
        Vector(name)
      } else if (sym.isType) {
        // it's type
        val name: String = s"type ${sym.name}"
        Vector(name)
      } else {
        // it's another symbol (binding, etd)
        Vector.empty
      }
    }

    tastys.foreach { tasty =>
      val path = tasty.path
      val tree: Tree = tasty.ast
      val symbol: Symbol = tree.symbol
      val result = processSymbol(symbol)

      println("-------------------------------")
      println(path)
      println()
      result.foreach(println)
    }
  }
}
object PrintInspector {

  def main(args: Array[String]): Unit = {
    TastyInspector.inspectTastyFiles(args.toList)(new PrintInspector)
  }
}

I wrote this quickly as a demo of the IDEa how to traverse Tasty using symbols API. When I compiled it with scala-cli and run against it's own tasty I got

# create Tasty files somewhere
> scala-cli compile .
Compiling project (Scala 3.2.1, JVM)
Compiled project (Scala 3.2.1, JVM)
# pass Tasty files from previpus compilation into the script to print itself
> scala-cli run PrintInspector.scala -- .scala-build/project_eb4be2fafa/classes/main/PrintInspector.tasty                                                                                     
Compiling project (Scala 3.2.1, JVM)
Compiled project (Scala 3.2.1, JVM)
-------------------------------
.scala-build/project_eb4be2fafa/classes/main/PrintInspector.class

package <empty>
  class PrintInspector
    def <init>
    def inspect
      val quotes
      val tastys
  val PrintInspector
  class PrintInspector$
    def <init>
    def writeReplace
    def main
      val args
    def apply

I believe this would be relatively easy for you to adapt to use extracted information to print case classes for each method or whatever you like.