0

I am working on a library that will take a user config hocon and user provide case class(in the classpath of the user application), I need to read the hocon config to the case class dynamically at runtime.. Dynamically I mean here that I would not know the case class during compile time, I would only know the case class at runtime by a user input.

How can I read the hocon into case class using its class name or ClassTag? I have tried using pureconfig or kxbmap/configs and other scala wrappers around tsconfig but these need the case class info at compile time.

Is there anyway or any tools to read hocon to case classes dynamically at runtime?

Praneeth Ramesh
  • 3,434
  • 1
  • 28
  • 34
  • 1
    Does the user action triggers the hocon file creation? And then you run a jar with this file config? – Tomer Shetah Nov 12 '20 at 07:42
  • 1
    There must be a better solution instead changing your app conf at runtime. Could your add more details? – Emiliano Martinez Nov 12 '20 at 09:30
  • 2
    I can only think of runtime reflection which recursively tries to initialize things and need to have some provided configs about handling "primitive" datatypes. But this really brings out the worst of Java: runtime reflection, runtime errors, annotations or text configs for configuring initialization etc. – Mateusz Kubuszok Nov 12 '20 at 10:24
  • @EmiCareOfCell44 User provides the static hocon files and also provide the case class to which they are read to, according to a naming convention.. I am writing a gradle task that will read the hocon files and parse them to case class objects. These case class objects are further used by plugin to create spark data frame and run some SQLs on it.. it’s not appconf you can think of it like static conf files(I would like to use hocon for its composable nature). – Praneeth Ramesh Nov 12 '20 at 16:14
  • Each time you receive a new hoocon-(case class) executes a gradle task?, but this is not runtime... what is the dynamic part?. You can read a hoocon dinamically but the case class con ept only makes sense at compile time. Besides, if you need to use spark sql the case class maybe it is not necessary (if you can extract the df schema from it). – Emiliano Martinez Nov 13 '20 at 09:27
  • The dynamic part is I dont know the case class or hocon I am parsing at compile time.. User will provide a hocon name like test.conf as input while invoking the gradle build and there will be a case class in user build classpath like Test.scala, I need to parse test.conf to Test.scala which I dont think I can derive implicit readers required at runtime since the there implicit readers are macros driven and I dont know the case class to parse at compile time.. – Praneeth Ramesh Nov 16 '20 at 02:39
  • Currently I am doing it using spark I am rendering the hocon to json and read the json with spark using spark.read.schema(caseClassSchema).json(json). Converting the CaseClass into sparkSchema (structType) and converting it into the Dataset[CaseClass] which works fine but I need to instantiate a sparkSession to do this.. I want to do it witthout sparkSession using natively hocon to case class parsers like pureconfig if possible. – Praneeth Ramesh Nov 16 '20 at 02:41
  • As @Mateusz Kubuszok suggested I tried recursively initialize case class, with which I need to handle too many primitive type conversions and I am still skeptical that it might hit some runtime exceptions.. – Praneeth Ramesh Nov 16 '20 at 02:47
  • @PraneethRamesh Here are several questions where classes are generated, implicits are resolved at runtime using Toolbox: https://stackoverflow.com/questions/64045433/load-dataset-from-dynamically-generated-case-class https://stackoverflow.com/questions/64308467/is-there-anyway-in-scala-to-get-the-singleton-type-of-something-from-the-more https://stackoverflow.com/questions/64102795/how-to-convert-json-to-scala-shapeless-hlist https://stackoverflow.com/questions/64798525/how-to-reflect-concrete-return-types-for-methods-of-classes-defined-at-runtime-u – Dmytro Mitin Nov 30 '20 at 01:34
  • @PraneethRamesh https://stackoverflow.com/questions/64903163/implicit-resolution-fail-in-reflection-with-toolbox https://stackoverflow.com/questions/56922911/dynamic-compilation-of-multiple-scala-classes-at-runtime https://stackoverflow.com/questions/60783153/tensorflow-in-scala-reflection https://stackoverflow.com/questions/64111895/dynamically-parse-json-in-flink-map – Dmytro Mitin Nov 30 '20 at 01:35
  • @PraneethRamesh Maybe approach with reflection or toolbox can work in your use case as well. Maybe I can help to write corresponding code (for class known at runtime) if firstly you show your code with hocon, pureconfig etc. for the case when a class is ordinary compile-time class. – Dmytro Mitin Nov 30 '20 at 01:39
  • @PraneethRamesh And even if implicits are macro-based this shouldn't in principle prevent their resolution at runtime if I understand correctly. See the question 64903163 out of above links. – Dmytro Mitin Nov 30 '20 at 01:49
  • @PraneethRamesh Did you manage to do what you wanted? – Dmytro Mitin Dec 13 '20 at 10:44

0 Answers0