How to compile and execute scala code at run-time in Scala3?

Question

I would like to compile and execute Scala code given as a String at run-time using Scala3. Like for example in Scala 2 I would have used Reflection

import scala.reflect.runtime.universe as ru
import scala.tools.reflect.ToolBox
val scalaCode = q"""println("Hello world!")"""
val evalMirror = ru.runtimeMirror(this.getClass.getClassLoader)
val toolBox = evalMirror.mkToolBox()
toolBox.eval(scalaCode) //Hello world!

If I try to run this code in Scala3 I get

Scala 2 macro cannot be used in Dotty. See https://dotty.epfl.ch/docs/reference/dropped-features/macros.html
To turn this error into a warning, pass -Xignore-scala2-macros to the compiler

How can I translate this code in Scala3 ?

Answer 1

Scala 2 version of this answer is here: How can I run generated code during script runtime?

In Scala 3:

• For example you can use Li Haoyi's Ammonite

ammonite.Main(verboseOutput = false).runCode("""println("Hello, World!")""")
// Hello, World!

build.sbt

scalaVersion := "3.1.3"
libraryDependencies += "com.lihaoyi" % "ammonite" % "2.5.4-22-4a9e6989" cross CrossVersion.full
excludeDependencies ++= Seq(
  ExclusionRule("com.lihaoyi", "sourcecode_2.13"),
  ExclusionRule("com.lihaoyi", "fansi_2.13"),
)

• Or you can try Eugene Yokota's Eval

com.eed3si9n.eval.Eval()
  .evalInfer("""println("Hello, World!")""")
  .getValue(this.getClass.getClassLoader)
// Hello, World!

build.sbt

scalaVersion := "3.2.0"
libraryDependencies += "com.eed3si9n.eval" % "eval" % "0.1.0" cross CrossVersion.full

• Or you can try my implementation

com.github.dmytromitin.eval.Eval[Unit]("""println("Hello, World!")""")
// Hello, World!

scalaVersion := "3.2.1"
libraryDependencies += "com.github.dmytromitin" %% "eval" % "0.1"

• Also you can use standard Scala 3 REPL interpreter

dotty.tools.repl.ScriptEngine().eval("""println("Hello, World!")""")
// Hello, World!

build.sbt

scalaVersion := "3.1.3"
libraryDependencies += scalaOrganization.value %% "scala3-compiler" % scalaVersion.value

• If you have a scala.quoted.Expr '{...} (a statically typed wrapper over an abstract syntax tree scala.quoted.Quotes#Tree) rather than plain string then you can use runtime multi-staging

import scala.quoted.*
given staging.Compiler = staging.Compiler.make(getClass.getClassLoader)
staging.run('{ println("Hello, World!") })
// Hello, World!

build.sbt

scalaVersion := "3.1.3"
libraryDependencies += scalaOrganization.value %% "scala3-staging" % scalaVersion.value

• All of the above is to run Scala 3 code in Scala 3. If we want to run Scala 2 code in Scala 3 then we can still use Scala 2 reflective Toolbox. Scala 2 macros don't work, so we can't do runtime.currentMirror or q"..." but can do universe.runtimeMirror or tb.parse

import scala.tools.reflect.ToolBox // implicit 

val tb = scala.reflect.runtime.universe
  .runtimeMirror(getClass.getClassLoader)
  .mkToolBox()
tb.eval(tb.parse("""println("Hello, World!")"""))
// Hello, World!

build.sbt

scalaVersion := "3.1.3"
libraryDependencies ++= scalaOrganization.value % "scala-compiler" % "2.13.8"

• Also to run Scala 2 code in Scala 3 you can use standard Scala 2 REPL interpreter

scala.tools.nsc.interpreter.shell.Scripted()
  .eval("""System.out.println("Hello, World!")""")
// Hello, World!

build.sbt

scalaVersion := "3.1.3"
libraryDependencies ++= scalaOrganization.value % "scala-compiler" % "2.13.8"

• Also you can use JSR223 scripting. Depending on whether you have scala3-compiler or scala-compiler in your classpath you will run Scala 3 or Scala 2 (one of the two above script engines: Scala 3 dotty.tools.repl.ScriptEngine or Scala 2 scala.tools.nsc.interpreter.shell.Scripted). If you have both the dependency added first wins.

new javax.script.ScriptEngineManager(getClass.getClassLoader)
  .getEngineByName("scala")
  .eval("""println("Hello, World!")""")
// Hello, World!

If you'd like to have a better control what dependency is used (without re-importing the project) you can use Coursier and specify class loader

import coursier.* // libraryDependencies += "io.get-coursier" %% "coursier" % "2.1.0-M6-53-gb4f448130" cross CrossVersion.for3Use2_13
val files = Fetch()
  .addDependencies(
    Dependency(Module(Organization("org.scala-lang"), ModuleName("scala3-compiler_3")), "3.2.0"),
    // Dependency(Module(Organization("org.scala-lang"), ModuleName("scala-compiler")), "2.13.9")
  )
  .run()

val classLoader = new java.net.URLClassLoader(
  files.map(_.toURI.toURL).toArray,
  /*getClass.getClassLoader*/null // ignoring current classpath
)
new javax.script.ScriptEngineManager(classLoader)
  .getEngineByName("scala")
  .eval("""
    type T = [A] =>> [B] =>> (A, B) // Scala 3
    //type T = List[Option[A]] forSome {type A} // Scala 2
    System.out.println("Hello, World!")
  """)
// Hello, World!

• You can implement Eval in Scala 3 yourself using actual compiler

import dotty.tools.io.AbstractFile
import dotty.tools.dotc.core.Contexts.Context
import dotty.tools.dotc.Driver
import dotty.tools.dotc.util.SourceFile
import dotty.tools.io.{VirtualDirectory, VirtualFile}
import java.net.URLClassLoader
import java.nio.charset.StandardCharsets
import dotty.tools.repl.AbstractFileClassLoader
import scala.io.Codec
import coursier.{Dependency, Module, Organization, ModuleName, Fetch}

  // we apply usejavacp=true instead
//  val files = Fetch()
//    .addDependencies(
//       Dependency(Module(Organization("org.scala-lang"), ModuleName("scala3-compiler_3")), "3.1.3"),
//    )
//    .run()
//
//  val depClassLoader = new URLClassLoader(
//    files.map(_.toURI.toURL).toArray,
//    /*getClass.getClassLoader*/ null // ignoring current classpath
//  )

val code =
  s"""
     |package mypackage
     |
     |object Main {
     |  def main(args: Array[String]): Unit = {
     |    println("Hello, World!")
     |  }
     |}""".stripMargin

val outputDirectory = VirtualDirectory("(memory)")
compileCode(code, List()/*files.map(f => AbstractFile.getFile(f.toURI.toURL.getPath)).toList*/, outputDirectory)
val classLoader = AbstractFileClassLoader(outputDirectory, this.getClass.getClassLoader/*depClassLoader*/)
runObjectMethod("mypackage.Main", classLoader, "main", Seq(classOf[Array[String]]), Array.empty[String])
// Hello, World!

def compileCode(
                 code: String,
                 classpathDirectories: List[AbstractFile],
                 outputDirectory: AbstractFile
               ): Unit = {
  class DriverImpl extends Driver {
    private val compileCtx0 = initCtx.fresh
    given Context = compileCtx0.fresh
      .setSetting(
        compileCtx0.settings.classpath,
        classpathDirectories.map(_.path).mkString(":")
      ).setSetting(
        compileCtx0.settings.usejavacp,
        true
      ).setSetting(
        compileCtx0.settings.outputDir,
        outputDirectory
      )
    val compiler = newCompiler
  }

  val driver = new DriverImpl
  import driver.given Context

  val sourceFile = SourceFile(VirtualFile("(inline)", code.getBytes(StandardCharsets.UTF_8)), Codec.UTF8)
  val run = driver.compiler.newRun
  run.compileSources(List(sourceFile))
  // val unit = run.units.head
  // println("untyped tree=" + unit.untpdTree)
  // println("typed tree=" + unit.tpdTree)
}

def runObjectMethod(
                     objectName: String,
                     classLoader: ClassLoader,
                     methodName: String,
                     paramClasses: Seq[Class[?]],
                     arguments: Any*
                   ): Any = {
  val clazz = Class.forName(s"$objectName$$", true, classLoader)
  val module = clazz.getField("MODULE$").get(null)
  val method = module.getClass.getMethod(methodName, paramClasses*)
  method.invoke(module, arguments*)
}

(previous version)

build.sbt

scalaVersion := "3.1.3"
libraryDependencies += scalaOrganization.value %% "scala3-compiler" % scalaVersion.value

See also: get annotations from class in scala 3 macros (hacking multi-staging programming in Scala 3 and implementing our own eval instead of Scala 2 context.eval or staging.run forbiden in Scala 3 macros).

• See also

An intro to the Scala Presentation Compiler

Parsing scala 3 code from a String into Scala 3 AST at runtime

Scala 3 Reflection

Help with dotty compiler and classloading at runtime