class not found error - scala

Question

i am running a scala program which is below. I am using maven for build and i have set the dependencies correctly and maven install was successful. But when am running the jar file i get java.lang.NoClassDefFoundError.

Program:

package RasterDataIngest.RasterDataIngestIntoHadoop

import geotrellis.spark._
import geotrellis.spark.ingest._
import geotrellis.spark.io.hadoop._
import geotrellis.spark.io.index._
import geotrellis.spark.tiling._
import geotrellis.spark.utils.SparkUtils
import geotrellis.vector._
import org.apache.hadoop.fs.Path
import org.apache.spark._
import com.quantifind.sumac.ArgMain
import com.quantifind.sumac.validation.Required

class HadoopIngestArgs extends IngestArgs {
  @Required var catalog: String = _
  def catalogPath = new Path(catalog)
}

object HadoopIngest extends ArgMain[HadoopIngestArgs] with Logging {
  def main(args: HadoopIngestArgs): Unit = {
   System.setProperty("com.sun.media.jai.disableMediaLib", "true")

    implicit val sparkContext = SparkUtils.createSparkContext("Ingest")
    val conf = sparkContext.hadoopConfiguration
    conf.set("io.map.index.interval", "1")

    val catalog = HadoopRasterCatalog(args.catalogPath)
    val source = sparkContext.hadoopGeoTiffRDD(args.inPath)
    val layoutScheme = ZoomedLayoutScheme()

    Ingest[ProjectedExtent, SpatialKey](source, args.destCrs, layoutScheme, args.pyramid){ (rdd, level) => 
      catalog
        .writer[SpatialKey](RowMajorKeyIndexMethod, args.clobber)
        .write(LayerId(args.layerName, level.zoom), rdd)
    }
  }
}

pom.xml:

<dependency>
  <groupId>org.scala-lang</groupId>
  <artifactId>scala-library</artifactId>
  <version>${scala.version}</version>
</dependency>
<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.10</artifactId>
  <version>1.5.2</version>
</dependency>
<dependency>
  <groupId>com.azavea.geotrellis</groupId>
  <artifactId>geotrellis-spark_2.10</artifactId> //this is the one
  <version>0.10.0-M1</version>
</dependency>
<dependency>
  <groupId>org.scalaz.stream</groupId>
  <artifactId>scalaz-stream_2.10</artifactId>
  <version>0.7.2a</version>
</dependency>
<dependency>
  <groupId>org.apache.hadoop</groupId>
  <artifactId>hadoop-core</artifactId>
  <version>0.20.2</version>
</dependency>
<dependency>
  <groupId>com.quantifind</groupId>
  <artifactId>sumac_2.10</artifactId>
  <version>0.3.0</version>
</dependency>

Error:

Exception in thread "main" java.lang.NoClassDefFoundError: geotrellis/spark/ingest/IngestArgs
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
        at java.lang.Class.getMethod0(Class.java:2866)
        at java.lang.Class.getMethod(Class.java:1676)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:670)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: geotrellis.spark.ingest.IngestArgs
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

please let me know where am going wrong.. Thanks beforehand!

score 0 · Accepted Answer · edited May 23 '17 at 12:31

0

It sounds like you are not building a jar which would actually contain your dependencies. If that is your problem, perhaps this answer would be helpful:

How can I create an executable JAR with dependencies using Maven?

edited May 23 '17 at 12:31

Community

1
1

answered Mar 04 '16 at 12:31

Teemu Ilmonen

316
1
5

score 0 · Answer 2 · answered Apr 01 '18 at 12:11

I encountered a similar issue and adding this to my pom.xml fixed it for me:

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>
  <configuration>
    <descriptorRefs>
      <descriptorRef>jar-with-dependencies</descriptorRef>
    </descriptorRefs>
  </configuration>
  <executions>
    <execution>
      <id>make-assembly</id>
      <phase>package</phase>
      <goals>
        <goal>single</goal>
      </goals>
    </execution>
  </executions>
</plugin>

This plugin helps build a JAR that contains all dependencies. Source: https://www.cloudera.com/documentation/enterprise/5-5-x/topics/spark_building.html#building

class not found error - scala

2 Answers2