0

So for spark sql you have do something like this:

val query = s"select *from table...."
val temp = sqlContext.sql(query)

Is there anything similar to this for just regular spark code? Like in python they have this exec(string) function that take string and run it as code. This is the link: Click Me

Anna
  • 443
  • 9
  • 29

1 Answers1

0

Gatling application is doing a similar trick, they allow you to write an arbitrary Scala code for load testing and then run it with their framework.

As far as I remember they basically write Scala code into a file, prepend/append it with something like:

class MyClass {
  def foo() = {
  // Code goes here
  }
}

and execute Scala compiler to compile this code and then load this code as a regular class.

Ivan Mushketyk
  • 8,107
  • 7
  • 50
  • 67