Java - Class exists twice (on classpath and on application jar). LinkageError: ClassCastException

Question

I am getting the following error when launching my Java Application.

I need to export some hadoop related directories to the classpath before launching the application to make it work (I can't skip this step), and I am getting the following error.

Caused by: java.lang.LinkageError: ClassCastException: attempting to castjar:file:/usr/hdp/3.0.0.0-1634/hadoop/lib/jsr311-api-1.1.1.jar!/javax/ws/rs/ext/RuntimeDelegate.class to jar:file:/tmp/blobStore-634df1c1-ffc8-4610-86af-8f39b33e4250/job_ac11246bea2bb31008c1a78212357514/blob_p-79f2d3193313ea987c15b4b28411db0fc2aa436c-f858cb54126b6d546c01e5ed453bf106!/javax/ws/rs/ext/RuntimeDelegate.class at javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:146) at javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:120) at javax.ws.rs.core.UriBuilder.newInstance(UriBuilder.java:95) at javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119) at org.glassfish.jersey.client.JerseyWebTarget.(JerseyWebTarget.java:71) at org.glassfish.jersey.client.JerseyClient.target(JerseyClient.java:290) at org.glassfish.jersey.client.JerseyClient.target(JerseyClient.java:76) at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.lambda$currentSchemaRegistryTargets$0(SchemaRegistryClient.java:293) at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.currentSchemaRegistryTargets(SchemaRegistryClient.java:293) at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getSupportedSchemaProviders(SchemaRegistryClient.java:384) at com.hortonworks.registries.schemaregistry.client.SchemaRegistryClient.getDefaultDeserializer(SchemaRegistryClient.java:969) at SchemaService.InitDeserializer(SchemaService.java:47) at SchemaService.deserialize(SchemaService.java:38) at org.apache.flink.streaming.connectors.kafka.internals.KafkaDeserializationSchemaWrapper.deserialize(KafkaDeserializationSchemaWrapper.java:45) at org.apache.flink.streaming.connectors.kafka.internal.KafkaFetcher.runFetchLoop(KafkaFetcher.java:140) at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.run(FlinkKafkaConsumerBase.java:712) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:93) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:57) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:97) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:302) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) at java.lang.Thread.run(Thread.java:745)

After making some research, I figured out that the class the program is trying to use is present in two diferent JAR files:

The first one is one transitive dependence on the libraries on my maven application (javax.ws.rs.jar)
The second one is a jar located in the directory with all the hadoop depdencies from hortonworks that I need to export into my classpath to make the application work (jsr311-api-1.1.1.jar)

So I need to tell the program in some way that the maven dependency packaged in the jar of my application should be used instead of the jar located on the classpath.(javax.ws.rs.jar) without removing this jar because it is part of my Big Data cluster installation and I can't play with those jars in the classpath.

Any thoughts?

pom.xml dependency causing the issue:

<dependency>
    <groupId>com.hortonworks.registries</groupId>
    <artifactId>schema-registry-client</artifactId>
    <version>${schema.registry.version}</version>
</dependency>

Dependency tree:

[INFO] amgrd:KafkaDemoProcessor:jar:0.0.1-SNAPSHOT
[INFO] +- org.apache.flink:flink-java:jar:1.8.1:compile
[INFO] |  +- org.apache.flink:flink-core:jar:1.8.1:compile
[INFO] |  |  +- org.apache.flink:flink-annotations:jar:1.8.1:compile
[INFO] |  |  +- org.apache.flink:flink-metrics-core:jar:1.8.1:compile
[INFO] |  |  +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile
[INFO] |  |  |  +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] |  |  |  \- org.objenesis:objenesis:jar:2.1:compile
[INFO] |  |  +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] |  |  \- org.apache.commons:commons-compress:jar:1.18:compile
[INFO] |  +- org.apache.flink:flink-shaded-asm:jar:5.0.4-6.0:compile
[INFO] |  +- org.apache.flink:flink-shaded-asm-6:jar:6.2.1-6.0:compile
[INFO] |  +- org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] |  +- org.apache.commons:commons-math3:jar:3.5:compile
[INFO] |  +- org.slf4j:slf4j-api:jar:1.7.15:compile
[INFO] |  +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] |  \- org.apache.flink:force-shading:jar:1.8.1:compile
[INFO] +- org.apache.flink:flink-streaming-java_2.12:jar:1.8.1:compile
[INFO] |  +- org.apache.flink:flink-runtime_2.12:jar:1.8.1:compile
[INFO] |  |  +- org.apache.flink:flink-queryable-state-client-java_2.12:jar:1.8.1:compile
[INFO] |  |  +- org.apache.flink:flink-hadoop-fs:jar:1.8.1:compile
[INFO] |  |  +- org.apache.flink:flink-shaded-netty:jar:4.1.32.Final-6.0:compile
[INFO] |  |  +- org.apache.flink:flink-shaded-jackson:jar:2.7.9-6.0:compile
[INFO] |  |  +- org.javassist:javassist:jar:3.19.0-GA:compile
[INFO] |  |  +- org.scala-lang:scala-library:jar:2.12.7:compile
[INFO] |  |  +- com.typesafe.akka:akka-actor_2.12:jar:2.4.20:compile
[INFO] |  |  |  +- com.typesafe:config:jar:1.3.0:compile
[INFO] |  |  |  \- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.8.0:compile
[INFO] |  |  +- com.typesafe.akka:akka-stream_2.12:jar:2.4.20:compile
[INFO] |  |  |  +- org.reactivestreams:reactive-streams:jar:1.0.0:compile
[INFO] |  |  |  \- com.typesafe:ssl-config-core_2.12:jar:0.2.1:compile
[INFO] |  |  |     \- org.scala-lang.modules:scala-parser-combinators_2.12:jar:1.0.4:compile
[INFO] |  |  +- com.typesafe.akka:akka-protobuf_2.12:jar:2.4.20:compile
[INFO] |  |  +- com.typesafe.akka:akka-slf4j_2.12:jar:2.4.20:compile
[INFO] |  |  +- org.clapper:grizzled-slf4j_2.12:jar:1.3.2:compile
[INFO] |  |  +- com.github.scopt:scopt_2.12:jar:3.5.0:compile
[INFO] |  |  +- org.xerial.snappy:snappy-java:jar:1.1.4:compile
[INFO] |  |  \- com.twitter:chill_2.12:jar:0.7.6:compile
[INFO] |  |     \- com.twitter:chill-java:jar:0.7.6:compile
[INFO] |  \- org.apache.flink:flink-shaded-guava:jar:18.0-6.0:compile
[INFO] +- org.apache.flink:flink-clients_2.12:jar:1.8.1:compile
[INFO] |  +- org.apache.flink:flink-optimizer_2.12:jar:1.8.1:compile
[INFO] |  \- commons-cli:commons-cli:jar:1.3.1:compile
[INFO] +- org.apache.flink:flink-connector-kafka_2.11:jar:1.8.1:compile
[INFO] |  +- org.apache.flink:flink-connector-kafka-base_2.11:jar:1.8.1:compile
[INFO] |  \- org.apache.kafka:kafka-clients:jar:2.0.1:compile
[INFO] |     \- org.lz4:lz4-java:jar:1.4.1:compile
[INFO] +- com.hortonworks.registries:schema-registry-client:jar:0.3.0:compile
[INFO] |  +- com.hortonworks.registries:schema-registry-common:jar:0.3.0:compile
[INFO] |  |  +- com.google.guava:guava:jar:18.0:compile
[INFO] |  |  +- org.slf4j:log4j-over-slf4j:jar:1.7.12:compile
[INFO] |  |  +- com.fasterxml.jackson.core:jackson-core:jar:2.7.3:compile
[INFO] |  |  \- com.fasterxml.jackson.core:jackson-databind:jar:2.7.3:compile
[INFO] |  +- org.glassfish.jersey.core:jersey-client:jar:2.22.1:compile
[INFO] |  |  +- javax.ws.rs:javax.ws.rs-api:jar:2.0.1:compile
[INFO] |  |  +- org.glassfish.jersey.core:jersey-common:jar:2.22.1:compile
[INFO] |  |  |  +- javax.annotation:javax.annotation-api:jar:1.2:compile
[INFO] |  |  |  +- org.glassfish.jersey.bundles.repackaged:jersey-guava:jar:2.22.1:compile
[INFO] |  |  |  \- org.glassfish.hk2:osgi-resource-locator:jar:1.0.1:compile
[INFO] |  |  +- org.glassfish.hk2:hk2-api:jar:2.4.0-b31:compile
[INFO] |  |  |  +- org.glassfish.hk2:hk2-utils:jar:2.4.0-b31:compile
[INFO] |  |  |  \- org.glassfish.hk2.external:aopalliance-repackaged:jar:2.4.0-b31:compile
[INFO] |  |  +- org.glassfish.hk2.external:javax.inject:jar:2.4.0-b31:compile
[INFO] |  |  \- org.glassfish.hk2:hk2-locator:jar:2.4.0-b31:compile
[INFO] |  +- org.glassfish.jersey.media:jersey-media-multipart:jar:2.22.1:compile
[INFO] |  |  \- org.jvnet.mimepull:mimepull:jar:1.9.6:compile
[INFO] |  +- com.hortonworks.registries:registry-common:jar:0.3.0:compile
[INFO] |  |  +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.7.3:compile
[INFO] |  |  |  \- org.yaml:snakeyaml:jar:1.15:compile
[INFO] |  |  +- com.github.fge:json-schema-validator:jar:2.2.6:compile
[INFO] |  |  |  +- joda-time:joda-time:jar:2.3:compile
[INFO] |  |  |  +- com.googlecode.libphonenumber:libphonenumber:jar:6.2:compile
[INFO] |  |  |  +- com.github.fge:json-schema-core:jar:1.2.5:compile
[INFO] |  |  |  |  +- com.github.fge:uri-template:jar:0.9:compile
[INFO] |  |  |  |  |  \- com.github.fge:msg-simple:jar:1.1:compile
[INFO] |  |  |  |  |     \- com.github.fge:btf:jar:1.2:compile
[INFO] |  |  |  |  +- com.github.fge:jackson-coreutils:jar:1.8:compile
[INFO] |  |  |  |  \- org.mozilla:rhino:jar:1.7R4:compile
[INFO] |  |  |  +- javax.mail:mailapi:jar:1.4.3:compile
[INFO] |  |  |  |  \- javax.activation:activation:jar:1.1:compile
[INFO] |  |  |  \- net.sf.jopt-simple:jopt-simple:jar:4.6:compile
[INFO] |  |  +- org.apache.hadoop:hadoop-client:jar:2.7.1:compile
[INFO] |  |  |  +- org.apache.hadoop:hadoop-common:jar:2.7.1:compile
[INFO] |  |  |  |  +- xmlenc:xmlenc:jar:0.52:compile
[INFO] |  |  |  |  +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] |  |  |  |  +- commons-codec:commons-codec:jar:1.4:compile
[INFO] |  |  |  |  +- commons-net:commons-net:jar:3.1:compile
[INFO] |  |  |  |  +- javax.servlet.jsp:jsp-api:jar:2.1:runtime
[INFO] |  |  |  |  +- commons-logging:commons-logging:jar:1.1.3:compile
[INFO] |  |  |  |  +- commons-lang:commons-lang:jar:2.6:compile
[INFO] |  |  |  |  +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] |  |  |  |  |  +- commons-digester:commons-digester:jar:1.8:compile
[INFO] |  |  |  |  |  |  \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] |  |  |  |  |  \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] |  |  |  |  +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] |  |  |  |  +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] |  |  |  |  +- org.apache.hadoop:hadoop-auth:jar:2.7.1:compile
[INFO] |  |  |  |  |  +- org.apache.httpcomponents:httpclient:jar:4.2.5:compile
[INFO] |  |  |  |  |  |  \- org.apache.httpcomponents:httpcore:jar:4.2.4:compile
[INFO] |  |  |  |  |  \- org.apache.curator:curator-framework:jar:2.7.1:compile
[INFO] |  |  |  |  +- org.apache.curator:curator-client:jar:2.7.1:compile
[INFO] |  |  |  |  +- org.apache.curator:curator-recipes:jar:2.7.1:compile
[INFO] |  |  |  |  +- org.apache.htrace:htrace-core:jar:3.1.0-incubating:compile
[INFO] |  |  |  |  \- org.apache.zookeeper:zookeeper:jar:3.4.6:compile
[INFO] |  |  |  |     \- io.netty:netty:jar:3.7.0.Final:compile
[INFO] |  |  |  +- org.apache.hadoop:hadoop-hdfs:jar:2.7.1:compile
[INFO] |  |  |  |  +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] |  |  |  |  +- io.netty:netty-all:jar:4.0.23.Final:compile
[INFO] |  |  |  |  +- xerces:xercesImpl:jar:2.9.1:compile
[INFO] |  |  |  |  |  \- xml-apis:xml-apis:jar:1.3.04:compile
[INFO] |  |  |  |  \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:compile
[INFO] |  |  |  \- org.apache.hadoop:hadoop-annotations:jar:2.7.1:compile
[INFO] |  |  +- org.hibernate:hibernate-validator:jar:5.1.3.Final:compile
[INFO] |  |  |  +- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] |  |  |  +- org.jboss.logging:jboss-logging:jar:3.1.3.GA:compile
[INFO] |  |  |  \- com.fasterxml:classmate:jar:1.0.0:compile
[INFO] |  |  \- org.springframework:spring-context:jar:4.3.6.RELEASE:compile
[INFO] |  |     +- org.springframework:spring-aop:jar:4.3.6.RELEASE:compile
[INFO] |  |     +- org.springframework:spring-beans:jar:4.3.6.RELEASE:compile
[INFO] |  |     +- org.springframework:spring-core:jar:4.3.6.RELEASE:compile
[INFO] |  |     \- org.springframework:spring-expression:jar:4.3.6.RELEASE:compile
[INFO] |  \- com.hortonworks.registries:common-auth:jar:0.3.0:compile
[INFO] |     \- org.apache.directory.server:apacheds-kerberos-codec:jar:2.0.0-M15:compile
[INFO] |        +- org.apache.directory.server:apacheds-i18n:jar:2.0.0-M15:compile
[INFO] |        +- org.apache.directory.api:api-asn1-api:jar:1.0.0-M20:compile
[INFO] |        \- org.apache.directory.api:api-util:jar:1.0.0-M20:compile
[INFO] \- com.hortonworks.registries:schema-registry-serdes:jar:0.3.0:compile
[INFO]    +- org.apache.avro:avro:jar:1.8.2:compile
[INFO]    |  +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO]    |  +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO]    |  +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
[INFO]    |  \- org.tukaani:xz:jar:1.5:compile
[INFO]    +- commons-io:commons-io:jar:2.5:compile
[INFO]    +- com.fasterxml.jackson.core:jackson-annotations:jar:2.7.3:compile
[INFO]    +- org.glassfish.jersey.media:jersey-media-json-jackson:jar:2.22.1:compile
[INFO]    |  \- org.glassfish.jersey.ext:jersey-entity-filtering:jar:2.22.1:compile
[INFO]    \- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.7.3:compile
[INFO]       +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.7.3:compile
[INFO]       \- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.3:compile

Thank you

score 1 · Answer 1 · answered Sep 20 '19 at 10:54

1

You can import like this if the class name will be same for resolving LinkageError.

So we have 2 classes with same name but in different package

First class

package com.bbc.config;

public class Test1 {
 }

And second class

package com.bbc.domain;

public class Test1 {
}

And the main class

package com.bbc.mainclass;
import com.bbc.config.Test1;

public class Main {
  public static void main(String[] args) {

  // this will import from config      
  Test1 test1 = new Test1(); // this will import from Test1

  // this will import from domain       
  com.bbc.domain.Test1 test11 = new com.bbc.domain.Test1();

  }
}

answered Sep 20 '19 at 10:54

Pawan Maurya

387
2
11

1

Thanks but In this case I am not writting the code who is importing the class, they are transitive dependencies of my libraries. The code is called inside the methods of the API's that I am ussing – dhalfageme Sep 20 '19 at 11:01
Okay, can you please paste the both dependencies here – Pawan Maurya Sep 20 '19 at 11:06
You need to change the scope in dependencies. https://howtodoinjava.com/maven/maven-dependency-scopes/ – Pawan Maurya Sep 20 '19 at 11:10
The dependences causing this error are schema-registry-client and schema-registry-serdes. I wrote the pom.xml above. But I do not need to exclude this on the pom.xml (if I do that, the program fails with ClassNotFound exception because the library is not included) – dhalfageme Sep 23 '19 at 08:18
I am not pretty sure that the error means what I am saying (trying to load jar in the classpath) since I am not used to work with Java, but it is what I understand here – dhalfageme Sep 23 '19 at 09:01
If I exclude "jerse-client" jar or "javax.ws.rs-api" in pom.xml to force use always the one in classpath, I get this exception then: .NoClassDefFoundError: org/glassfish/jersey/client/ClientConfig – dhalfageme Sep 23 '19 at 10:31

score 1 · Accepted Answer · answered Sep 24 '19 at 09:55

I add these jar on classpath with these version. And build the and run. I don't find any cause.

<!-- https://mvnrepository.com/artifact/com.hortonworks.registries/schema-registry-client -->
        <dependency>
            <groupId>com.hortonworks.registries</groupId>
            <artifactId>schema-registry-client</artifactId>
            <version>0.8.0</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.hortonworks.registries/schema-registry-serdes -->
        <dependency>
            <groupId>com.hortonworks.registries</groupId>
            <artifactId>schema-registry-serdes</artifactId>
            <version>0.8.0</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.glassfish.jersey.core/jersey-client -->
        <dependency>
            <groupId>org.glassfish.jersey.core</groupId>
            <artifactId>jersey-client</artifactId>
            <version>2.29.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/javax.ws.rs/javax.ws.rs-api -->
        <dependency>
            <groupId>javax.ws.rs</groupId>
            <artifactId>javax.ws.rs-api</artifactId>
            <version>2.1</version>
        </dependency>

I added those dependencies an also (groupId: org.glassfish.jersey.inject, artifactId: jersey-hk2 trying to fix the following exception that I'm getting now, but even adding this dependency the exception remains: "java.lang.IllegalStateException: InjectionManagerFactory not found. at org.glassfish.jersey.internal.inject.Injections.lambda$lookupInjectionManagerFactory$0(Injections.java:74) — dhalfageme, Sep 24 '19 at 10:51
https://stackoverflow.com/questions/44088493/jersey-stopped-working-with-injectionmanagerfactory-not-found Please check this resource — Pawan Maurya, Sep 24 '19 at 11:45
I've tried with bot jersey-hk2 and jersey-cdi2-se dependencies. For jersey-hk2 I get the exception mentioned above, and for jersey-cdi2-se, I tried now and this is the error I get .IllegalStateException: WELD-ENV-000016: Missing beans.xml file in META-INF. — dhalfageme, Sep 24 '19 at 12:31
It worked with jersey-hk2 dependency, I don't know why the first time I tried it failed. Thanks! — dhalfageme, Sep 26 '19 at 08:39

Java - Class exists twice (on classpath and on application jar). LinkageError: ClassCastException

2 Answers2