dynamically parse a string and return a function in scala using reflection and interpretors

Question

I am trying to dinamically interpret code given as a String. Eg:

val myString = "def f(x:Int):Int=x+1".

Im looking for a method that will return the real function out of it: Eg:

val myIncrementFunction = myDarkMagicFunctionThatWillBuildMyFunction(myString)
println(myIncrementFunction(3))

will print 4

Use case: I want to use some simple functions from that interpreted code later in my code. For example they can provide something like def fun(x: Int): Int = x + 1 as a string, then I use the interpreter to compile/execute that code and then I'd like to be able to use this fun(x) in a map for example.

The problem is that that function type is unknown for me, and this is one of the big problems because I need to cast back from IMain. I've read about reflection, type system and such, and after some googling I reached this point. Also I checked twitter's util-eval but I cant see too much from the docs and the examples in their tests, it's pretty the same thing.

If I know the type I can do something like

val settings = new Settings
val imain = new IMain(settings)
val res = imain.interpret("def f(x:Int):Int=x+1; val ret=f _ ")
val myF = imain.valueOfTerm("ret").get.asInstanceOf[Function[Int,Int]]
println(myF(2))

which works correctly and prints 3 but I am blocked by the problem I said above, that I dont know the type of the function, and this example works just because I casted to the type I used when I defined the string function for testing how IMain works.

Do you know any method how I could achieve this functionality ?

I'm a newbie so please excuse me if I wrote any mistakes.

Thanks

score 2 · Answer 1 · edited Oct 23 '19 at 15:26

You can use twitter-util library to do this, check the test file: https://github.com/twitter/util/blob/b0696d0/util-eval/src/test/scala/com/twitter/util/EvalTest.scala

If you need to use IMain, maybe because you want to use the intepreter with your own custom settings, you can do something like this:

a. First create a class meant to hold your result:

    class ResHolder(var value: Any)

b. Create a container object to hold the result and interpret the code into that object:

    val settings = new Settings()
    val writer = new java.io.StringWriter()
    val interpreter = new IMain(settings, writer)

    val code = "def f(x:Int):Int=x+1"

    // Create a container object to hold the result and bind in the interpreter
    val holder = new ResHolder(null) 

    interpreter.bind("$result", holder.getClass.getName, holder) match {
       case Success => 
       case Error => throw new ScriptException("error in: binding '$result' value\n" + writer)
       case Incomplete => throw new ScriptException("incomplete in: binding '$result' value\n" + writer)
    }

    val ir = interpreter.interpret("$result.value = " + code)

    // Return cast value or throw an exception based on result
    ir match {
       case Success =>
          val any = holder.value
          any.asInstanceOf[(Int) => Int]

       case Error => throw new ScriptException("error in: '" + code + "'\n" + writer)
       case Incomplete => throw new ScriptException("incomplete in :'" + code + "'\n" + writer)
    }

score 2 · Accepted Answer · answered Jan 07 '16 at 22:08

Ok, I managed to achieve the functionality I wanted, I am still looking for improving this code, but this snippet does what I want.

I used scala toolbox and quasiquotes

import scala.reflect.runtime.universe.{Quasiquote, runtimeMirror}
import scala.tools.reflect.ToolBox

object App {
    def main(args: Array[String]): Unit = {
        val mirror = runtimeMirror(getClass.getClassLoader)
        val tb = ToolBox(mirror).mkToolBox()

        val data = Array(1, 2, 3)

        println("Data before function applied on it")
        println(data.mkString(","))


        println("Please enter the map function you want:")
        val function = scala.io.StdIn.readLine()
        val functionWrapper = "object FunctionWrapper { " + function + "}"
        val functionSymbol = tb.define(tb.parse(functionWrapper).asInstanceOf[tb.u.ImplDef])

        // Map each element using user specified function
        val dataAfterFunctionApplied = data.map(x => tb.eval(q"$functionSymbol.function($x)"))

        println("Data after function applied on it")
        println(dataAfterFunctionApplied.mkString(","))
    }
}

And here is the result in the terminal:

Data before function applied on it
1,2,3
Please enter the map function you want:
def function(x: Int): Int = x + 2
Data after function applied on it
3,4,5

Process finished with exit code 0

Just something to add. Also, terrific answer, a bit more context can be found at http://docs.scala-lang.org/overviews/quasiquotes/usecases. To convert to an apply-able function use something like: val data = tb.eval(q"$functionSymbol.f _").asInstanceOf[Int => Int] — Andrew Scott Evans, May 16 '16 at 19:36
Toolbox is consuming a lot of time to parse and eval string to code — Tarun Khaneja, Oct 24 '18 at 06:01
Yeah that's right. Actually never really used this, just for some tutorial I did at that time — Doru Chiulan, Aug 11 '22 at 13:12

score 2 · Answer 3 · answered Oct 30 '17 at 16:22

I wanted to elaborate the previous answer with the comment and perform an evaluation of the solutions:

import scala.reflect.runtime.universe.{Quasiquote, runtimeMirror}
import scala.tools.reflect.ToolBox

object Runtime {

  def time[R](block: => R): R = {
    val t0 = System.nanoTime()
    val result = block    // call-by-name
    val t1 = System.nanoTime()
    println("Elapsed time: " + (t1 - t0) + " ns")
    result
  }

    def main(args: Array[String]): Unit = {
        val mirror = runtimeMirror(getClass.getClassLoader)
        val tb = ToolBox(mirror).mkToolBox()
        val data = Array(1, 2, 3)

        println(s"Data before function applied on it: '${data.toList}")
        val function = "def apply(x: Int): Int = x + 2"
        println(s"Function: '$function'")
        println("#######################")

        // Function with tb.eval
        println(".... with tb.eval")
        val functionWrapper = "object FunctionWrapper { " + function + "}"
        // This takes around 1sec!
        val functionSymbol = time { tb.define(tb.parse(functionWrapper).asInstanceOf[tb.u.ImplDef])}

        // This takes around 0.5 sec!
        val result = time {data.map(x => tb.eval(q"$functionSymbol.apply($x)"))}
        println(s"Data after function applied on it: '${result.toList}'")

        println(".... without tb.eval")
        val func = time {tb.eval(q"$functionSymbol.apply _").asInstanceOf[Int => Int]}
        // This takes around 0.5 sec!
        val result2 = time {data.map(func)}
        println(s"Data after function applied on it: '${result2.toList}'")

    }
}

If we execute the code above we see the following output:

Data before function applied on it: 'List(1, 2, 3)
Function: 'def apply(x: Int): Int = x + 2'
#######################
.... with tb.eval
Elapsed time: 716542980 ns
Elapsed time: 661386581 ns
Data after function applied on it: 'List(3, 4, 5)'
.... without tb.eval
Elapsed time: 394119232 ns
Elapsed time: 85713 ns
Data after function applied on it: 'List(3, 4, 5)'

Just to emphasize the importance of do the evaluation to extract a Function, and then apply to the data, without the end to evaluate again, as the comment in the answer indicates.

dynamically parse a string and return a function in scala using reflection and interpretors

3 Answers3

Linked