Replace a transitive dependency building maven fat jar

Question

I am trying to replace a transitive dependency with a latest one while creating my fat jar. But every time the old dependency gets included in the jar. I have tried both assembly plugin as well as shade plugin. Here is snippet from my pom-

<dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
        <version>2.3.1</version>
        <scope>compile</scope>
        <exclusions>
            <exclusion>
                <groupId>org.apache.kafka</groupId>
                <artifactId>kafka_2.11</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.11</artifactId>
        <version>0.10.2.0</version>
    </dependency>

Shade plugin snippet-

<plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.0.0</version>
            <configuration>
                <artifactSet>
                    <excludes>
                        <exclude>org.apache.kafka:kafka-clients:*</exclude>
                    </excludes>
                </artifactSet>
                <filters>
                    <filter>
                        <artifact>*:*</artifact>
                        <excludes>
                            <exclude>META-INF/*.SF</exclude>
                            <exclude>META-INF/*.DSA</exclude>
                            <exclude>META-INF/*.RSA</exclude>
                        </excludes>
                    </filter>
                </filters>
            </configuration>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

Any help will be appreciated.

Have you tried the `dependency:tree` command to see who's linking to the wrong version? That's the easiest way to track these things down. Then you can `exclude` the transitive dependency. — jwismar, Mar 12 '19 at 17:33
Yes, I did! Spark-streaming-kafka-0-10_2.11 is adding that dependency but even after excluding the transitive dependency from this dependency the transitive dependency still gets packaged into the fat jar. I have already shown how I am doing it. — Y0gesh Gupta, Mar 12 '19 at 17:56

score 1 · Answer 1 · answered Mar 13 '19 at 20:41

Try reordering the dependency.

Keep the dependency with the version you want on top of the dependency from where it is getting included through transitive dependency.

Do mvn dependency:tree a few times to get it right.

Example:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.11</artifactId>
    <version>0.10.2.0</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
    <version>2.3.1</version>
    <scope>compile</scope>
    <!-- <exclusions>
    <exclusion>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.11</artifactId>
        </exclusion>
    </exclusions> -->
</dependency>

Ref:

1. https://stackoverflow.com/questions/31740785/why-order-of-maven-dependencies-matter
2. https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Transitive_Dependencies

Replace a transitive dependency building maven fat jar

1 Answers1