0

I am trying to run a drools based (and KieServices based) project from a fat jar that has been generated using sbt assembly.

[main] INFO org.drools.compiler.kie.builder.impl.ClasspathKieProject - Found kmodule: jar:file:/.../myJar.jar!/META-INF/kmodule.xml
[main] ERROR org.drools.compiler.kie.builder.impl.ClasspathKieProject - Unable to build index of kmodule.xml url=jar:file:/.../myJar.jar/META-INF/kmodule.xml
You're trying to perform a xml related operation without the necessary xml support for drools. Please add the module org.drools:drools-xml-support to your classpath.
[main] ERROR org.drools.compiler.kie.builder.impl.KieContainerImpl - Unknown KieSession name: DroolDummyKS

This error is triggered by the following code, last line to be specific:

val kieServices: KieServices = KieServices.Factory.get
val kieContainer: KieContainer = kieServices.getKieClasspathContainer

// Apply the correct kie session from the ./resources/META-INF/kmodule.xml configuration
val kieSession: KieSession = kieContainer.newKieSession("DroolDummyKS")

Project is a Scala SBT project (Java 11 as compiler)

GSerum_
  • 57
  • 8

1 Answers1

2

I couldn't reproduce Unable to build index of kmodule.xml. Just kieContainer.newKieSession("DroolDummyKS") returns null for assembly jar (java -jar myJar.jar), on contrary to sbt run, where it returns KieSession[0].

Notice that it's written in your error:

Please add the module org.drools:drools-xml-support to your classpath

Do two things in build.sbt:

  • add to libraryDependencies

    "org.drools" % "drools-xml-support" % "8.31.1.Final" 
    

and

  • unignore kmodule.xml in assembly strategy (so that kmodule.xml is included into assembly jar), for example with singleOrError (not sure that concatenation makes sense for xml on contrary to service files, we're making sure that it's your kmodule.xml being included, otherwise it throws)

    assembly / assemblyMergeStrategy := {
      case PathList("META-INF", "services", xs@_*) => MergeStrategy.concat
      case PathList("META-INF", "kmodule.xml") => MergeStrategy.singleOrError
      case PathList("META-INF", xs@_*) => MergeStrategy.discard
      case _ => MergeStrategy.first
    }
    

Update. With

assembly / assemblyMergeStrategy := {
  case _ => MergeStrategy.singleOrError
}

you can check what files have duplicates.

You should add the dependency "org.drools" % "drools-xml-support" % "8.31.1.Final" as I adviced earlier. Otherwise there is NPE even for sbt run. I see that you added it in update.

You should remove the file src/main/resources/META-INF/services/org.kie.api.KieServices. Anyway it's present in the dependency drools-compiler-8.31.1.Final.jar.

Try the strategy ignoring as less as possible

assembly / assemblyMergeStrategy := {
  case xs if xs.endsWith("LICENSE") => MergeStrategy.discard
  case xs if xs.endsWith("LICENSE.txt") => MergeStrategy.discard
  case xs if xs.endsWith("INDEX.LIST") => MergeStrategy.discard
  case xs if xs.endsWith("MANIFEST.MF") => MergeStrategy.discard
  case xs if xs.endsWith("NOTICE") => MergeStrategy.discard
  case xs if xs.endsWith("NOTICE.txt") => MergeStrategy.discard
  case xs if xs.endsWith("module-info.class") => MergeStrategy.discard
  case PathList("META-INF", "services", "org.apache.poi.sl.draw.ImageRenderer") => MergeStrategy.filterDistinctLines
  case PathList("META-INF", "services", "org.apache.poi.ss.usermodel.WorkbookProvider") => MergeStrategy.filterDistinctLines
  case PathList("META-INF", "services", "org.apache.poi.extractor.ExtractorProvider") => MergeStrategy.filterDistinctLines
  case PathList("META-INF", "services", "org.drools.wiring.api.ComponentsSupplier") => MergeStrategy.filterDistinctLines
  case _ => MergeStrategy.singleOrError
}

Duplicates should be resolved then.

filterDistinctLines is similar to concat, just not adding the same lines.

The strategy can be simplified

assembly / assemblyMergeStrategy := {
  case xs if Seq(
    "LICENSE",
    "LICENSE.txt",
    "INDEX.LIST",
    "MANIFEST.MF",
    "NOTICE",
    "NOTICE.txt",
    "module-info.class"
  ).exists(xs.endsWith) => MergeStrategy.discard
  case PathList("META-INF", "services", xs@_*) => MergeStrategy.filterDistinctLines
  case _ => MergeStrategy.singleOrError
}

Update 2. I looked for reasons of the latest NPE

Exception in thread "main" java.lang.NullPointerException: 
Cannot invoke 
"org.drools.compiler.compiler.Dialect.getId()"
because the return value of
"org.drools.compiler.rule.builder.RuleBuildContext.getDialect()"
is null

It turns out that the thing was in the file META-INF/kie.default.properties.conf of one of dependencies. So it was enough to additionally unignore it

assembly / assemblyMergeStrategy := {
  case x if x.endsWith("module-info.class") => MergeStrategy.discard
  case PathList("META-INF", "services", xs@_*) => MergeStrategy.concat
  case PathList("META-INF", "kmodule.xml") => MergeStrategy.singleOrError
  case PathList("META-INF", "kie.default.properties.conf") => MergeStrategy.singleOrError
  case PathList("META-INF", xs@_*) => MergeStrategy.discard
  case _ => MergeStrategy.first
}

But the main conclusion for us should be that ignoring the whole META-INF can be dangerous. There can be some other files there that are important for some of dependencies. Maybe now there is no NPE but some issues can be later.

It turns out that the default assembly strategy

assembly / assemblyMergeStrategy := MergeStrategy.defaultMergeStrategy

or

assembly / assemblyMergeStrategy := {
  case x =>
    val oldStrategy = (ThisBuild / assemblyMergeStrategy).value
    oldStrategy(x)
}

or (see here)

val defaultMergeStrategy: String => MergeStrategy = {
  case x if Assembly.isConfigFile(x) =>
    MergeStrategy.concat
  case PathList(ps @ _*) if Assembly.isReadme(ps.last) || Assembly.isLicenseFile(ps.last) =>
    MergeStrategy.rename
  case PathList("META-INF", xs @ _*) =>
    (xs map {_.toLowerCase}) match {
      case ("manifest.mf" :: Nil) | ("index.list" :: Nil) | ("dependencies" :: Nil) =>
        MergeStrategy.discard
      case ps @ (x :: xs) if ps.last.endsWith(".sf") || ps.last.endsWith(".dsa") =>
        MergeStrategy.discard
      case "plexus" :: xs =>
        MergeStrategy.discard
      case "services" :: xs =>
        MergeStrategy.filterDistinctLines
      case ("spring.schemas" :: Nil) | ("spring.handlers" :: Nil) =>
        MergeStrategy.filterDistinctLines
      case _ => MergeStrategy.deduplicate
    }
  case _ => MergeStrategy.deduplicate
}

does the work pretty well. You just had to ignore additionally files module-info.class. So you can prefer

assembly / assemblyMergeStrategy := {
  case x if x.endsWith("module-info.class") => MergeStrategy.discard
  case x =>
    val oldStrategy = (ThisBuild / assemblyMergeStrategy).value
    oldStrategy(x)
}

https://github.com/ThijmenL98/DroolsMCVE/pull/1

Dmytro Mitin
  • 48,194
  • 3
  • 28
  • 66
  • Whilst this does fix something, it doesn't completely get rid of errors on this line: `Exception in thread "main" java.lang.ExceptionInInitializerError Caused by: java.lang.NullPointerException at org.drools.compiler.rule.builder.RuleBuildContext.` – GSerum_ Dec 15 '22 at 15:27
  • @GSerum_ On what line?? You quoted exception but didn't provide the code which leads to the exception. Please provide the reproduction. – Dmytro Mitin Dec 15 '22 at 15:35
  • Same line, the newKieSession call – GSerum_ Dec 15 '22 at 15:42
  • @GSerum_ Can't reproduce. `val kieSession: KieSession = kieContainer.newKieSession("DroolDummyKS")` `println(kieSession)` prints `KieSession[0]` both for `sbt run` and `java -jar myJar.jar`. How to reproduce this current your NPE? Try to debug what exactly is `null`. – Dmytro Mitin Dec 15 '22 at 15:46
  • @GSerum_ Don't forget to do `sbt clean` before `sbt run`, `sbt package`, `sbt assembly` to avoid cached something outdated. – Dmytro Mitin Dec 15 '22 at 15:49
  • @GSerum_ How do you run `myJar.jar`? Is it `java -jar myJar.jar`? It seems you had `Unable to build index of kmodule.xml` (which I couldn't reproduce either) earlier in sbt: `[main] INFO ... [main] ERROR ...` Tried to use `myJar.jar` as an (unmanaged) dependency in a new project, put it into `lib`, anyway `sbt run` prints `KieSession[0]`. Can't reproduce your latest NPE. – Dmytro Mitin Dec 15 '22 at 16:03
  • @GSerum_ Could you try not to ignore `META-INF` at all? `assembly / assemblyMergeStrategy := { case _ => MergeStrategy.first }` or `assembly / assemblyMergeStrategy := { case _ => MergeStrategy.last }` – Dmytro Mitin Dec 15 '22 at 16:17
  • @GSerum_ If I remove `assembly / assemblyMergeStrategy` at all I can see that the only file with duplicates is `META-INF/versions/9/module-info.class` in several dependencies. Please see https://stackoverflow.com/questions/54834125/sbt-assembly-deduplicate-module-info-class *"If you use JDK 11 and create a library then it is better not to create an uber-jar. In this case a dropping of the `module-info.class` will most likely create a jar that will not work. In this case simply depend on the libraries."* Why do you need assembly jar? – Dmytro Mitin Dec 15 '22 at 16:22
  • @GSerum_ Try `assembly / assemblyMergeStrategy := { case x if x.endsWith("module-info.class") => MergeStrategy.discard; case x => val oldStrategy = (assembly / assemblyMergeStrategy).value; oldStrategy(x) }` – Dmytro Mitin Dec 15 '22 at 16:29
  • Thank you for your replies, I will try this tomorrow and will come bakc to you – GSerum_ Dec 15 '22 at 16:49
  • @GSerum_ `oldStrategy` is to use default sbt-assembly strategy, it's here: https://github.com/sbt/sbt-assembly/blob/v0.15.0/src/main/scala/sbtassembly/MergeStrategy.scala#L126-L150 – Dmytro Mitin Dec 15 '22 at 16:54
  • @GSerum_ By the way, isn't sbt-assembly 0.15.0 too old? Current is 2.1.0. – Dmytro Mitin Dec 15 '22 at 16:59
  • I need an uber jar because it is creating a CLI application executable jar. The update to assembly 2.1.0 and your updated merge strategy did not change the exception unfortunately. – GSerum_ Dec 19 '22 at 05:44
  • I indeed run my jar as`java -jar myJar.jar`, and get the following error: `[main] INFO org.drools.compiler.kie.builder.impl.InternalKieModuleProvider - Creating KieModule for artifact org.default:artifact:1.0.0 - [main] INFO org.drools.modelcompiler.CanonicalKieModuleProvider - No executable model found for artifact org.default:artifact:1.0.0. Falling back to resources parsing. - [main] INFO org.drools.compiler.kie.builder.impl.KieContainerImpl - Start creation of KieBase: DroolDummyKB - Exception in thread "main" java.lang.ExceptionInInitializerError` – GSerum_ Dec 19 '22 at 05:52
  • @GSerum_ Please provide step-by-step reproduction for your current error. Where those `[main] INFO ...` come from? From a logger? I can't see logging dependency in the `build.sbt` from your previous question. So apparently your description of the current question doesn't reflect current state of your project. Please update the question with the current state. For example you can publish somewhere (at Github?) your sources and jars, write step-by-step reproduction (what you're doing in details, what messages and errors you have) etc. – Dmytro Mitin Dec 19 '22 at 11:46
  • I use the SL4J logger in my project but don't have it associated with drools or KIE, i think it just creates its own or something. – GSerum_ Dec 19 '22 at 14:34
  • @GSerum_ Maybe I can't reproduce your error because of some differences in our projects. I published sources at https://github.com/DmytroMitin/drools-kie-demo-assembly and assembly jar at https://oss.sonatype.org/content/groups/public/com/github/dmytromitin/drools-kie-demo-assembly_2.13/ I used the assembly jar as a dependency at https://scastie.scala-lang.org/DmytroMitin/QNhi1lWzSZCHoNwMtAPrpw/1 As you can see there is no exception. So either try to find difference between our projects or publish yours. Not much can be done without reproduction. – Dmytro Mitin Dec 19 '22 at 21:37
  • @GSerum_ If you download the assembly jar from https://oss.sonatype.org/content/groups/public/com/github/dmytromitin/drools-kie-demo-assembly_2.13/0.1/drools-kie-demo-assembly_2.13-0.1.jar what does `java -jar drools-kie-demo-assembly_2.13-0.1.jar` produce for you? `KieSession[0]` or exception? – Dmytro Mitin Dec 20 '22 at 01:50
  • @GSerum_ Scastie uses Java 17 but with [scala-cli](https://scala-cli.virtuslab.org/) and Java 11 `scala-cli repl --scala 2.13.10 --jvm 11 --dep com.github.dmytromitin::drools-kie-demo-assembly:0.1` `scala> src.Main.main(Array[String]())` also produces `KieSession[0]`. – Dmytro Mitin Dec 20 '22 at 03:33
  • Indeed `KieSession[0]` – GSerum_ Dec 20 '22 at 11:39
  • @GSerum_ So there is some difference in projects. Please provide reproduction if you need further assistance. Would you publish your project? – Dmytro Mitin Dec 20 '22 at 12:48
  • 1
    I'll see what I can do, I cannot share too much as it's protected code, so i need to try my best :) – GSerum_ Dec 20 '22 at 18:19
  • @GSerum_ Ok, good luck. If you have reproduction, ping me. By the way, an alternative to assembly would be `publishLocal` and using sbt launcher https://www.scala-sbt.org/1.x/docs/Launcher-Getting-Started.html https://stackoverflow.com/questions/74440324/sbt-gives-java-lang-nullpointerexception-when-trying-to-run-spark (chapter "What is the launcher"). You can use your project as a dependency without fat jar. – Dmytro Mitin Dec 21 '22 at 05:48
  • 1
    I am still unable to get it to work, created a repo for you here: https://github.com/ThijmenL98/DroolsMCVE @DmytroMitin – GSerum_ Jan 03 '23 at 08:08
  • Yeah this one properly reproduces my issue, running Main works fine, but when using assembly and running the jar using `java -jar ./target/scala-2.12/myJar.jar` (or `scala-2.13` can't remember), it throws a nullpointer. – GSerum_ Jan 03 '23 at 15:51
  • 1
    @GSerum_ See one more update about the reasons of NPE and that the default assembly strategy does almost what you need – Dmytro Mitin Jan 04 '23 at 13:59
  • 1
    @GSerum_ Just noticed that I recommended to try this default strategy in my comment [Dec 15, 2022 at 16:29](https://stackoverflow.com/questions/74809158/run-drools-kie-project-from-fat-jar#comment132035100_74809294) :) – Dmytro Mitin Jan 04 '23 at 14:23
  • 1
    Interesting link https://github.com/sbt/sbt-onejar – Dmytro Mitin Jan 04 '23 at 14:49
  • 1
    Thank you Dmytro, I have run into a new issue once again, if you're interested, this time it has to do with testing :) https://stackoverflow.com/questions/75028579/drools-time-continuous-drools-rules-not-working – GSerum_ Jan 06 '23 at 08:30