Kryo library has "overlapping classes" warning in maven shade-package phase

Question

I am trying to create a shaded jar file but the shade plugin is giving me warnings. I am not sure how to fix them.

Here is my pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>example.shade.group</groupId>
    <artifactId>myproj-shade-kryo</artifactId>
    <name>myproj-shade-kryo</name>
    <version>1.0.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>com.esotericsoftware.kryo</groupId>
            <artifactId>kryo</artifactId>
            <version>2.21</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-enforcer-plugin</artifactId>
                <executions>
                    <execution>
                        <id>enforce</id>
                        <configuration>
                            <rules>
                                <DependencyConvergence/>
                            </rules>
                        </configuration>
                        <goals>
                            <goal>enforce</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.4.2</version>
                <executions>
                    <execution>
                        <id>package</id>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

And the warning is:

[INFO] --- maven-shade-plugin:2.4.2:shade (package) @ myproj-shade-kryo ---
[INFO] Including com.esotericsoftware.kryo:kryo:jar:2.21 in the shaded jar.
[INFO] Including com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07 in the shaded jar.
[INFO] Including org.ow2.asm:asm:jar:4.0 in the shaded jar.
[INFO] Including com.esotericsoftware.minlog:minlog:jar:1.2 in the shaded jar.
[INFO] Including org.objenesis:objenesis:jar:1.2 in the shaded jar.
[WARNING] minlog-1.2.jar, kryo-2.21.jar define 2 overlapping classes: 
[WARNING]   - com.esotericsoftware.minlog.Log
[WARNING]   - com.esotericsoftware.minlog.Log$Logger
[WARNING] reflectasm-1.07-shaded.jar, kryo-2.21.jar define 23 overlapping classes: 
[WARNING]   - com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.Opcodes
[WARNING]   - com.esotericsoftware.reflectasm.AccessClassLoader
[WARNING]   - com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.Label
[WARNING]   - com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.ClassWriter
[WARNING]   - com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.AnnotationVisitor
[WARNING]   - com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.Type
[WARNING]   - com.esotericsoftware.reflectasm.FieldAccess
[WARNING]   - com.esotericsoftware.reflectasm.ConstructorAccess
[WARNING]   - com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.ClassVisitor
[WARNING]   - com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.Edge
[WARNING]   - 13 more...
[WARNING] objenesis-1.2.jar, kryo-2.21.jar define 32 overlapping classes: 
[WARNING]   - org.objenesis.ObjenesisBase
[WARNING]   - org.objenesis.instantiator.gcj.GCJInstantiator
[WARNING]   - org.objenesis.ObjenesisHelper
[WARNING]   - org.objenesis.instantiator.jrockit.JRockitLegacyInstantiator
[WARNING]   - org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator
[WARNING]   - org.objenesis.instantiator.ObjectInstantiator
[WARNING]   - org.objenesis.instantiator.gcj.GCJInstantiatorBase$DummyStream
[WARNING]   - org.objenesis.instantiator.basic.ObjectStreamClassInstantiator
[WARNING]   - org.objenesis.ObjenesisException
[WARNING]   - org.objenesis.Objenesis
[WARNING]   - 22 more...
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/

I tried the mvn dependency:tree -Ddetail=true too but that does not give me any clues how to fix:

[INFO] ------------------------------------------------------------------------
[INFO] Building myproj-shade-kryo 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ myproj-shade-kryo ---
[INFO] example.shade.group:myproj-shade-kryo:jar:1.0.0-SNAPSHOT
[INFO] \- com.esotericsoftware.kryo:kryo:jar:2.21:compile
[INFO]    +- com.esotericsoftware.reflectasm:reflectasm:jar:shaded:1.07:compile
[INFO]    |  \- org.ow2.asm:asm:jar:4.0:compile
[INFO]    +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO]    \- org.objenesis:objenesis:jar:1.2:compile
[INFO] ------------------------------------------------------------------------

Well for example `reflectasm-1.07-shaded.jar` and `kryo-2.21.jar` both bring the "same" classes, i.e. same fully qualified name. So you need to decide which one you exclude to fix this. If they come from transitive dependency, exclude it. Otherwise, select the content with [include/exclude](https://maven.apache.org/plugins/maven-shade-plugin/examples/includes-excludes.html). — Tunaki, Nov 03 '16 at 17:10
Wont this be a problem if I exclude one of the dependencies based on lower/higher version alone? — user2250246, Nov 03 '16 at 17:22
I think the right behavior here should be the ability to repackage dependencies which have different versions. So foo.bar.HelloWorld should become foo.bar.one.HelloWorld for version 1 and foo.bar.two.HelloWorld for version 2. For dependencies with same version, its not a problem anyways. — user2250246, Nov 03 '16 at 17:24
(Thinking aloud) It seems very strange that `reflectasm-1.07-shaded.jar` and `kryo-2.21.jar` both define classes in the same package with the same name. Wouldn't `kryo-2.21` have noticed this when doing their own builds? Same classes with same package names as the dependencies should be very rare IMHO. Example: Why would kryo declare `org.objenesis.ObjenesisBase` in their own code base? — user2250246, Nov 03 '16 at 17:32
I'm afraid only the developers of that library could answer that one... — Tunaki, Nov 03 '16 at 17:38
@radistao I don't believe this is a duplicate. Kryo is doing something very unusual for a maven build: they're including classes from a completely different open source project (objenesis) but not shading/renaming them. — Brad Cupit, Mar 10 '17 at 16:02
you are right: removed duplication flag and added description to the answer: http://stackoverflow.com/a/42628444/907576 — radistao, Mar 11 '17 at 10:10

score 0 · Answer 1 · edited May 23 '17 at 12:31

It was a problem in kryo-2.21, seems to be fixed in kryo-2.22.

Behind the scene:

kryo-2.21 package contains:

com
META-INF
org/objenesis/*
minlog-1.2.jar
objenesis-1.2.jar
reflectasm-1.07-shaded.jar

and objenesis-1.2.jar itself contains:

META-INF
org/objenesis/*

So, both root kryo and inner objenesis-1.2.jar archives contain org/objenesis/* classes!

Same for reflectasm library: it is included as the classes directly to kryo/com/* and as a jar kryo/reflectasm-1.07-shaded.jar

This seems to be fixed in kryo-2.22, guess this minimizeJar build property makes the effect

For other cases how to exclude overlapping from multiple libraries see Maven build [WARNING] we have a duplicate class

Also some times (if you are sure there is no overlapping classes in your dependencies) - try to clean the project:

mvn clean package

Kryo library has "overlapping classes" warning in maven shade-package phase

1 Answers1