My question is very similiar to the one found here: NoClassDefFoundError Apache Storm I have tried all the solutions posted there and I do not think the issue is the same.
I can compile my java code to a .jar without issue, but when I run it I get the below error. I included all the dependencies in the pom, yet it still can not find it. The code is almost directly copied from the Apache Storm documentation, to remove any other aspects that might give issues.
Any ideas would be greatly appreciated.
I get the following error message when trying to run my topology on my storm cluster:
Running: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/home/hadoop/storm -Dstorm.log.dir=/home/hadoop/stormLog -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file= -cp /home/hadoop/storm/*:/home/hadoop/storm/lib-worker/*:/home/hadoop/storm/extlib/*:twitterTest2.jar:/home/hadoop/storm/conf:/home/hadoop/storm/bin: -Dstorm.jar=twitterTest2.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} processingT.TTopology
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/storm/kafka/spout/KafkaSpout
at processingT.TTopology.run(TTopology.java:24)
at org.apache.storm.topology.ConfigurableTopology.start(ConfigurableTopology.java:68)
at processingT.TTopology.main(TTopology.java:16)
Caused by: java.lang.ClassNotFoundException: org.apache.storm.kafka.spout.KafkaSpout
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 3 more
I have this in my pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.storm.processing</groupId>
<artifactId>processingT</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-client</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka-client</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-kafka</artifactId>
<version>1.2.3</version>
</dependency>
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-server</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.13</artifactId>
<version>2.6.0</version>
<exclusions>
<exclusion>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</project>
And the java code is:
package processingT;
import org.apache.storm.topology.ConfigurableTopology;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.kafka.spout.KafkaSpout;
import org.apache.storm.kafka.spout.KafkaSpoutConfig;
import org.apache.storm.tuple.Fields;
public class TTopology extends ConfigurableTopology {
public static void main(String[] args) throws Exception {
ConfigurableTopology.start(new TTopology(), args);
}
@Override
protected int run(String[] args) throws Exception {
String port = "9092";
TopologyBuilder tp = new TopologyBuilder();
tp.setSpout("kafka_spout", new KafkaSpout<>(KafkaSpoutConfig.builder("ip:" + port, "topic").build()), 1);
tp.setBolt("countT", new CountTBolt(), 12).fieldsGrouping("kafka_spout", new Fields("word"));
conf.setDebug(true);
String topologyName = "count";
conf.setNumWorkers(3);
if (args != null && args.length > 0) {
topologyName = args[0];
}
return submit(topologyName,conf,tp);
}
}