How to resolve split package issue in Java 9

Question

Issue

I have module-info.java file that looks like:

module server {
 
   ...
   // split package issue: lot of java classes within packages with the same name
   requires hbase.common;
   requires hbase.client;
   ...
}

Workaround 1

Problem can't be solved by maven-shader-plugin, because it doesn't know groupId, only package name. Means shader will rename the same packages from both: hbase.common and hbase.client - split package problem remains.

Workaround 2

I also tried to create some shader middlelayer module, to throw away unneeded packages and resolve split package issue. But this solution also does not work.

shader/module-info.java:

module shader {
    requires hbase.common;
    // exports only packages I do need at my code. Shade unneded packages
    // IS THERE ANY WAY TO MAKE IT WORK?
    // Got: X module reads package org.apache.hadoop.hbase.util from both shader and hbase.common
    exports org.apache.hadoop.hbase.util;
}

server/module-info.java

module server {
    requires shader;
    requires hbase.client;
}

PS

Are there any Maven plugins for combining split-package jars?

(1) If you don't mind headaches --> https://stackoverflow.com/questions/42358084/package-conflicts-with-automatic-modules-in-java-9 (2) otherwise, remove your `module-info.java` file until those dependencies are jigsaw-compatible. — assylias, Jul 25 '18 at 14:46
Can you add to the details, what version of hbase client, common or others are you using? — Naman, Jul 25 '18 at 14:52
@assylias the reference does not provide an answer. Unsplit two jars is not an option, I'm not going to rewrite hbase. Merge two jars is not a solution two, because it's not manageble/updateble — VB_, Jul 25 '18 at 14:54
@nullpointer both `hbase.client` and `hbase.common` of version 1.2.6. — VB_, Jul 25 '18 at 14:56
@assylias I can not throw away Java9's modules, because I need OSGI like behaviour. Means I have conflict of transitive dependencies between hadood and jetcd, and I hoped to solve them by using modules — VB_, Jul 25 '18 at 15:02
Maybe a not good news for you https://hbase.apache.org/book.html#java — Naman, Jul 25 '18 at 15:03
@nullpointer I believe it's not compatible due split packages problem) It works if I delete `module-info.java` file — VB_, Jul 25 '18 at 15:12
@VolodymyrBakhmatiuk Just thinking out loud, one way of doing that(merging jars) could be to package them in another project([maven](https://stackoverflow.com/questions/25591991/combine-multiple-jar-into-one-using-maven)) then use that artifact as a dependency to your actual modularised code. (As said, I haven't given this a try either. But then you can [figure this out by generating module-info using jdeps](https://stackoverflow.com/questions/47500529/missing-dependencies-when-generate-module-info-jdeps) if the package exported are sorted or not.) — Naman, Jul 25 '18 at 17:28
@nullpointer could you please take a look at my question update (ps section)? — VB_, Jul 26 '18 at 09:23
@VolodymyrBakhmatiuk There is one functional doubt in the approach, how did you arrive at the conclusion to resolve the ambiguity in favor of `hbase-common` and not `hbase-client`? I hope you get what I am asking. Though this is similar to what I was suggesting in my previous comment as well. But yeah, I would say too unreliable to be used in any sensitive/deployable code. — Naman, Jul 26 '18 at 11:48
@nullpointer I need `hbase-client` for all code besides `org.apache.hadoop.hbase.util` package. For that package I need `hbase-common`. This is obvious from my code, because I can see which classes are imported. The package names are identica, but classes are different. That means I can easily figure out which package from which JAR I actually need — VB_, Jul 26 '18 at 12:07
@VolodymyrBakhmatiuk The problem is not that simple. It arises when the same package exposes different classes from different modules and both of them are required. Anyway, glad that you've got at least one way of solving the split package using maven. — Naman, Jul 26 '18 at 13:24
@nullpointer yeah, you're right. My approach does not solve the problem when the same package exposes different classes from different modules and both of them are required. — VB_, Jul 26 '18 at 13:39

VB_ · Answer 1 · 2018-07-26T13:40:14.627

Important this approach does not work when the same package exposes different classes from different modules and both of them are required. It only works if different packages are used, so you can filter-out packages from conflicting JARs.

So, the problem - two dependencies (maybe transitive) with the same package name. It's not compatible with JMPS and failfast during compilation. The solution of this problem is to manually (with maven-shade-plugin)exclude conflicting packages from one of the dependencies.

maven-shade-plugin has per-class or per-package include/exclude functionality. Here is documentation.

Not working solution (explanation of the problem)

The problem was that this approach doesn't work at first glance. If you will put the plugin at the same pom.xml where you imported both conflicting JARs - compilation will fail due "module X reads package org.apache.hadoop.hbase.util from both hbase.client and hbase.common ". JPMS is run at compilation phase (before package phase where plugin is launched). Here is an example:

server/
  |-src/
  |  |-main/
  |     |-java/
  |        |-com.somapackage
  |        |-module-info.java
  |           [ 
  |               module server {
  |                   requires java.base;
  |                   requires hbase.common; //they have lots of conflicting packages
  |                   requires hbase.client; //they have lots of conflicting packages
  |               }
  |            ]
  |-pom.xml
      [
    <dependencies>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-common</artifactId>
            <version>${hbase.common.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
         </dependency>
         <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>${hbase.client.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
         </dependency>
    </dependencies>
    <build>
    <plugins>
           <plugin>
               <groupId>org.apache.maven.plugins</groupId>
               <artifactId>maven-shade-plugin</artifactId>
               <version>3.1.1</version>
               <executions>
                   <execution>
                       <phase>package</phase>
                       <goals>
                           <goal>shade</goal>
                       </goals>
                       <configuration>
                           <minimizeJar>false</minimizeJar>
                           <filters>
                               <filter>
                                   <artifact>org.apache.hbase:hbase-client</artifact>
                                   <excludes>
                                       <exclude>org/apache/hadoop/hbase/util/ **</exclude>
                                   </excludes>
                               </filter>
                               <filter>
                                   <artifact>org.apache.hbase:hbase-common</artifact>
                                   <includes>
                                       <include>org/apache/hadoop/hbase/util/ **</include>
                                   </includes>
                               </filter>
                           </filters>
                       </configuration>
                   </execution>
               </executions>
           </plugin>
       </plugins>
       </build>
      ]

Working solution

To make JPMS split package validation work after shading we have to move conflicting dependencies to separate submodule. Moreover, we have to manually resolve packages conflict (exclude the same package from one dependency and include into another dependency) - that means we have to create to different sumbodules: shader1 and shader2. Here is the code:

server/
  |-src/
  |  |-main/
  |     |-java/
  |        |-com.somapackage
  |        |-module-info.java
  |           [ 
  |               module server {
  |                   requires java.base;
  |                   requires shader1;
  |                   requires shader2;
  |               }
  |            ]
  |-pom.xml
      [
    <dependency>
        <groupId>com.organization.proj</groupId>
        <artifactId>shader1</artifactId>
        <version>${proj.version}</version>
        <scope>system</scope>
        <systemPath>${project.basedir}/../shader/target/shader1-0.0.1.jar</systemPath>
    </dependency>
    <dependency>
        <groupId>com.organization.proj</groupId>
        <artifactId>shader2</artifactId>
        <version>${proj.version}</version>
        <scope>system</scope>
        <systemPath>${project.basedir}/../shader2/target/shader2-0.0.1.jar</systemPath>
    </dependency>
      ]

  |
  |
shader1
  |-src/
  |  |-main/
  |     |-java/
  |        |-com.somapackage
  |        |-module-info.java
  |           [ 
  |               module shader1 {
  |                   requires java.base;
  |                   requires transitive hbase.client;
  |               }
  |            ]
  |-pom.xml
      [
    <dependencies>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>${hbase.client.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
         </dependency>
    </dependencies>
    <build>
    <plugins>
           <plugin>
               <groupId>org.apache.maven.plugins</groupId>
               <artifactId>maven-shade-plugin</artifactId>
               <version>3.1.1</version>
               <executions>
                   <execution>
                       <phase>package</phase>
                       <goals>
                           <goal>shade</goal>
                       </goals>
                       <configuration>
                           <minimizeJar>false</minimizeJar>
                           <filters>
                               <filter>
                                   <artifact>org.apache.hbase:hbase-client</artifact>
                                   <excludes>
                                       <exclude>org/apache/hadoop/hbase/util/ **</exclude>
                                   </excludes>
                               </filter>
                           </filters>
                       </configuration>
                   </execution>
               </executions>
           </plugin>
       </plugins>
       </build>
      ]
  |
  |
shader2
  |-src/
  |  |-main/
  |     |-java/
  |        |-com.somapackage
  |        |-module-info.java
  |           [ 
  |               module shader2 {
  |                   requires java.base;
  |                   requires transitive hbase.common;
  |               }
  |            ]
  |-pom.xml
      [
    <dependencies>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-common</artifactId>
            <version>${hbase.common.version}</version>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
         </dependency>
    </dependencies>
    <build>
    <plugins>
           <plugin>
               <groupId>org.apache.maven.plugins</groupId>
               <artifactId>maven-shade-plugin</artifactId>
               <version>3.1.1</version>
               <executions>
                   <execution>
                       <phase>package</phase>
                       <goals>
                           <goal>shade</goal>
                       </goals>
                       <configuration>
                           <minimizeJar>false</minimizeJar>
                           <filters>
                               <filter>
                                   <artifact>org.apache.hbase:hbase-common</artifact>
                                   <includes>
                                       <include>org/apache/hadoop/hbase/util/ **</include>
                                   </includes>
                               </filter>
                           </filters>
                       </configuration>
                   </execution>
               </executions>
           </plugin>
       </plugins>
       </build>
      ]

do not forget to use require transitive at shader submodules