-3

Can I map Scala functions to JSON; or perhaps via a different way than JSON?

I know I can map data types, which is fine. But I'd like to create a function, map it to JSON send it via a REST method to another server, then add that function to a list of functions in another application and apply it.

For instance:

def apply(f: Int => String, v: Int) = f(v)

I want to make a list of functions that can be applied within an application, over different physical locations. Now I want to add and remove functions to the list. By means of REST calls.

Let's assume I understand security problems...

ps.. If you downvote, you might as well have the decency to explain why

Danielson
  • 2,605
  • 2
  • 28
  • 51
  • Well... the answer is both Yes and No. As for why, then you need to understand what a function actually is. A function is not like `data` which can be transferred by "marshalling". A function defines a process in the environment, which means that if the process defined by the function is tightly coupled (function with side effects) with the environment it-self, then it can not be transferred to some other environment. Even for the functions which do not have side effect, you have to ensure that `context` of the functional closure is "clean" (side effect free). – sarveshseri Apr 27 '18 at 08:47
  • Above two are minimal conditions for transferability of the function, rest of the explanation will be too involved for a question on stack overflow. – sarveshseri Apr 27 '18 at 08:50
  • @Danielson what do you mean by " I'd like to create a function, map it to JSON"? Wouldn't you create a `val` and map it? You would map the result of a function, not the function itself. – James Whiteley Apr 27 '18 at 09:01
  • @SarveshKumarSingh Right... then I'll just send a scala file (as text), compile that then load the functions. I still don't see why that is not possible... – Danielson Apr 27 '18 at 09:02
  • 1
    Also the Scala Playframework has a handy `Json.toJson` function for converting case classes to JSON. Might be worth looking into that. – James Whiteley Apr 27 '18 at 09:03
  • @JamesWhiteley I want some services that do stuff... Each service will have a list a functions that they can apply to certain sets of values. I want these services to exchange functions - not data... I mentioned JSON, because that's what I'm used to working with. – Danielson Apr 27 '18 at 09:03
  • @JamesWhiteley case classes, eh? Interesting... :-) – Danielson Apr 27 '18 at 09:06
  • Well IMO, I would never suggest passing raw code between services. You are just asking for malicious exploits. – James Whiteley Apr 27 '18 at 09:06
  • @JamesWhiteley I understand securities problems ;-) for starters, these kinds of projects are not open-sourced, nor will the algorithms be visible. It is mainly to check the boundaries of what can be done – Danielson Apr 27 '18 at 09:08
  • 1
    I'll answer with an example of how to change case classes to JSON. I would still personally never pass raw code/functions between services. – James Whiteley Apr 27 '18 at 09:09
  • @Danielson Security by obscurity is no security at all, so being closed source does not make a system secure. – Tim Apr 27 '18 at 09:41

3 Answers3

2

If I understand correctly, you want to be able to send Scala code to be executed on different physical machines. I can think of a few different ways of achieving that

  1. Using tools for distributed computing e.g. Spark. You can set up Spark clusters on different machines and then select to which cluster you want to submit Spark jobs. There are a lot of other tools for distributed computing that might also be worth looking into.
  2. Pass scala code as a string and compile it either within your server side code (here's an example) or by invoking scalac as an external process.
  3. Send the functions as byte code and execute the byte code on the remote machine.

If it fits with what you want to do, I'd recommend option #1.

Make sure that you can really trust the code that you want to execute, to not expose yourself to malicious code.

Simon
  • 6,293
  • 2
  • 28
  • 34
  • Thanks for first answering my question, and ending with a mild precaution. 1) interesting... I doubt whether that's favourable, since all `actors` need specific tasks.. Nonetheless, I must look into that. 2) that seems better than REPL 3) is this like 2, but just byte[] instead of String? 4) precaution, I will ;-) – Danielson Apr 27 '18 at 09:20
1

The answer is you can't do this, and even if you could you shouldn't!

You should never, never, never write a REST API that allows the client to execute arbitrary code in your application.

What you can do is create a number of named operations that can be executed. The client can then pass the name of the operation which the server can look up in a Map[String, <function>] and execute the result.

Tim
  • 26,753
  • 2
  • 16
  • 29
0

As mentioned in my comment, here is an example of how to turn a case class into JSON. Things to note: don't question the implicit val format line (it's magic); each case class requires a companion object in order to work; if you have Optional fields in your case class and define them as None when turning it into JSON, those fields will be ignored (if you define them as Some(whatever), they will look like any other field). If you don't know much about Scala Play, ignore the extra stuff for now - this is just inside the default Controller you're given when you make a new Project in IntelliJ.

package controllers

import javax.inject._
import play.api.libs.json.{Json, OFormat}
import play.api.mvc._

import scala.concurrent.Future

@Singleton
class HomeController @Inject()(cc: ControllerComponents) extends AbstractController(cc) {

  case class Attributes(heightInCM: Int, weightInKG: Int, eyeColour: String)

  object Attributes {
    implicit val format: OFormat[Attributes] = Json.format[Attributes]
  }

  case class Person(name: String, age: Int, attributes: Attributes)

  object Person {
    implicit val format: OFormat[Person] = Json.format[Person]
  }

  def index: Action[AnyContent] = Action.async {

    val newPerson = Person("James", 24, Attributes(192, 83, "green"))

    Future.successful(Ok(Json.toJson(newPerson)))

  }

}

When you run this app with sbt run and hit localhost:9000 through a browser, the output you see on-screen is below (formatted for better reading). This is also an example of how you might send JSON as a response to a GET request. It isn't the cleanest example but it works.

{
    "name":"James",
    "age":24,
    "attributes":
    {
        "heightInCM":187,
        "weightInKG":83,
        "eyeColour":"green"
    }
}

Once more though, I would never recommend passing actual functions between services. If you insist though, maybe store them as a String in a Case Class and turn it into JSON like this. Even if you are okay with passing functions around, it might even be a good exercise to practice security by validating the functions you receive to make sure they're not malicious.

I also have no idea how you'll convert them back into a function either - maybe write the String you receive to a *.scala file and try to run them with a Bash script? Idk. I'll let you figure that one out.

James Whiteley
  • 3,363
  • 1
  • 19
  • 46
  • He does not want to send "data". What he wants to do is to "send the function" to be executed somewhere else. This answer is not related to the question in any way. – sarveshseri Apr 27 '18 at 09:34
  • As I said in my answer, if you wanted to send a function like this you would store it as a String in a case class and convert it to JSON this way. As I also said in my answer, this is a response to a comment I made on the original post about converting a case class to JSON as that seemed to be something OP was interested in. This is a direct response to their comments... – James Whiteley Apr 27 '18 at 09:38
  • @james Thanks for the answer. I thought you meant that functions could be added to a case class. But what Sarvesh says is true... It's about functions... I will delete my question when the next downvote comes... It seems that people see down votes, and rush to vote - good for your sense of worth and medals. I haven't heard a good reason why this is not a valid question... – Danielson Apr 27 '18 at 09:39
  • 1
    This question and all in it is going quite wrong... Perhaps I should have just asked: "why is there a nullpointerexception" in my code. It feels like this site has be deteriorating the last couple of years. People down-vote because they don't understand (then the OP must be wrong) or because there is already a down-vote, so the question must be bad...... James and Sarvash, I appreciate your help :-) – Danielson Apr 27 '18 at 09:41
  • 1
    @Denielson I understand that this is a question in your mind. But the thing is that this question can not be answered in a short discussion. Even if someone can answer it and you manage to understand, the solution will involve too much work for to be able to handle any real world function. I am certain that you do not want to spend next 4-5 months in building just the framework which will enable you to do this. And even then the said framework will come with its own limitations. – sarveshseri Apr 27 '18 at 09:53
  • 1
    @Danielson Firstly, my apologies if I have contributed to any of the problems. This is a genuine comment so please do not take it as criticism but an attempt to help: You asked a very specific question about marshalling an existing Scala `function` object over JSON in the same way that you can marshal a data object. This is a perfectly valid question and has, I think, been answered by multiple people. However the discussion expanded to more general issues of sending executable code between servers, leading to confusion over what you were actually asking. I think this is what led to downvotes. – Tim Apr 27 '18 at 10:25
  • @SarveshKumarSingh good point... I did expect it to be simpler, like just an annotation I missed. – Danielson Apr 27 '18 at 10:39
  • @Tim I don't know. I couldn't ask the question any broader, because then (I know) it is not a `good` question, but perhaps I've made to too specific (into the wrong direction). I sort of expected negativity before posting the question. It is quite easy to create medium-high valued questions. But meaningful ones are often too hard for mainstream to understand (based on multiple questions), hence the idea 'I don't understand it, so you must be wrong'. But no worries... :-) I don't care about the reputation, but I do dislike the sheep-like voting (never the fault of one; basic group behaviour). – Danielson Apr 27 '18 at 10:41