0

I'm trying to dynamically create a chain of functions to perform on a numeric value. The chain is created at runtime from text instructions. The trick is that the functions vary in what types they produce. Some functions produce a Double, some produce a Long.

Update

The core issue is that I have a massive amount of data to process, but different values require different processing. In addition to the data I have specifications on how to extract and manipulate values to their final form, such as applying a polynomial, using a lookup table, changing the binary format (like 2s Compliment), etc. These specs are in a file of some sort (I'm creating the file form a database, but that's not important to the conversation), and I can apply these specs to multiple data files.

so with functions (these are just exmaples; there are tons of them):

  def Multiply(input: Long, factor:Double):Double = input*factor
  def Poly(input:Double, co:Array[Double]):Double = // do some polynomial math

I can manually create a chain like this:

val poly = (x: Double) => EUSteps.Poly(x,Array[Double](1,2))
val mult = (x: Long) => EUSteps.Multiply(x, 1.5)
val chain = mult andThen poly 

And if I call chain(1) I get 4

Now I want to be able to parse a string like "MULT(1.5);POLY(1,2)" and get that same chain. The idea is that I can define the chain however I want. Maybe its "MULT(1.5);MULT(2);POLY(1,2,3)." for example. So I can make the functions generic, like this:

  def Multiply[A](value: A, factor:Double)(implicit num: Numeric[A]) = num.toDouble(value)*factor
  def Poly[A](value:A, co:Array[Double])(implicit num: Numeric[A]) = { // do some poly math

Parsing the string isn't hard as it's very simple.
How can I build the chain dynamically? If it helps, the input is always going to be Long for the first step in the chain. The result could be Long or Double, and I'm OK with it if I have to do two versions based on the end result, so one that goes Long to Long, the other that goes Long to Double.

What I've tried

If I define my functions as having the same signature, like this:

  def Multiply(value: Double, factor:Double) = value*factor
  def Poly(value:Double, co:Array[Double]) = {

I can do it as part of a map operation:

  def ParseList(instruction:String) = {
    var instructions = instruction.split(';')

    instructions.map(inst => {
      val instParts = inst.split(Array(',','(',')'))
      val instruction = instParts(0).toUpperCase()
      val instArgs = instParts.drop(1).map(arg => arg.toDouble)
      instruction match {
        case "POLY" => (x: Double) => EUSteps.Poly(x,instArgs)
        case "MULTI" => (x: Double) => Multiply(x,instArgs(0))
      }
    }).reduceLeft((a,b) => a andThen b)

However, that breaks as soon as I change one of the arguments or return types to Long:

  def Multiply(value: Long, factor:Double) = value*factor

And change my case

      instruction match {
        case "POLY" => (x: Double) => EUSteps.Poly(x,instArgs)
        case "MULTI" => (x: Long) => Multiply(x,instArgs(0))
      }
    }).reduceLeft((a,b) => a andThen b)

Now the Reduce is complaining because it wanted Double => Double instead of Long => Double

Update 2

The way I solved it was to do what Levi suggested in the comments. I'm sure this is not very Scala-y, but when in doubt I go back to my OO roots. I suspect there is a more elegant way to do it though.

I declared an abstract class called ParamVal:

abstract class ParamVal {
  def toDouble(): Double

  def toLong(): Long
}

Then Long and Double types to go with it that implement the conversions:

case class DoubleVal(value: Double) extends ParamVal {
  override def toDouble(): Double = value

  override def toLong(): Long = value.toLong
}
case class LongVal(value: Long) extends ParamVal {
  override def toDouble(): Double = value.toDouble

  override def toLong(): Long = value
}

This lets me define all function inputs as ParamVal, and since each one expects a certain input type it's easy to just call toDouble or toLong as needed. NOTE: The app that creates these instructions already makes sure the chain is correct.

jpwkeeper
  • 321
  • 2
  • 10
  • 1
    You basically want a parser and an evaluator since you are in essence writing a programming language. My personal advice is to always try to avoid this, if your users will have to program then make them program. – Luis Miguel Mejía Suárez Feb 05 '23 at 15:46
  • No, I'm not. I'm translating from an existing app that lets you process the values in various ways that the user defines in a GUI. The string itself is arbitrary; I could do it in any form I want, I just figured that was the easiest to develop and debug using a string. I'm controlling both ends of this. – jpwkeeper Feb 05 '23 at 16:27
  • 1
    Yes, you are, just not a big general-purpose language like **Java** but a domain-specific one, the fact that it is graphical is also irrelevant, your users are programming. I am not saying that what you are doing is wrong, I am rather pointing you to the right direction, you want a parser and an interpreter. – Luis Miguel Mejía Suárez Feb 05 '23 at 16:32
  • You misunderstand. The user has already defined how to process the data. That's an app that's been in existence for 15 years. I'm migrating the processing capability to Spark. Nobody is typing this in as code. I could do it as binary instructions just as easily, but that's harder to debug, especially at the beginning. How I get the instructions over isn't important, it's how I create the chain dynamically. – jpwkeeper Feb 05 '23 at 16:35
  • Are you having a working monomorphic (without `[A]`) code (parser/evaluator) and trying to make it polymorphic (with `[A: Numeric]`)? It's hard to say how to make it polymorphic without actual code. What is your specific problem with making it polymorphic? – Dmytro Mitin Feb 06 '23 at 02:21
  • 1
    I mean, a trivial answer to *"How can I build the chain dynamically"* would be "similarly to the way you did it before". It's not clear what specifically prevents you from doing that. – Dmytro Mitin Feb 06 '23 at 02:26
  • 1
    @DmytroMitin because as soon as I try to do this as an array functions I have to have all of the functions in the array have the same argument and return type. Because the value goes from Long to Double and back through the chain, while I can do it manually, I can't do it as part of a list operation after parsing. – jpwkeeper Feb 06 '23 at 11:07
  • I've expanded my question to include the approach I've tried. Perhaps that will help understand where I'm headed. – jpwkeeper Feb 06 '23 at 11:23
  • 1
    Have you tried defining a `Value` supertrait with `DoubleValue` and `LongValue` etc. subclasses? Then the functions to execute are all `(Value, Value) => Value`. You might want to implement something to type-check the tree before building the linear chain of functions. – Levi Ramsey Feb 06 '23 at 11:33
  • https://stackoverflow.com/questions/3508077/how-to-define-type-disjunction-union-types – Dmytro Mitin Feb 07 '23 at 05:43
  • 1
    @LeviRamsey That's basically what I did. I'll update my question with what I did. It's essentially a bit more explicit than the Either solution. – jpwkeeper Feb 16 '23 at 11:21
  • In general, when implementing a programming language, the runtime representation is lower-level/less-typed than the programming language itself or even the language the implementation is in. On the CPU, it's all bits/bytes/words with no real types anyway (ignoring things like FPU registers and such). As James Mickens might put it, "your CPU isn't going to learn types by osmosis from putting the writings of Wadler on it". – Levi Ramsey Feb 16 '23 at 16:34

1 Answers1

1

Some ideas:

  1. Analyze the string chain upfront and figure out what will be the type of the final result and then use it for all steps all along. You will need a family of functions for each type.

  2. Try to use Either[Long, Double] in the reduce part.

michaJlS
  • 2,465
  • 1
  • 16
  • 22