Error: Cannot load main class from JAR file:/loudacre/kb/KBDOC-00299.html Spark Error

Question

I have used the answer available in the following StackOverflow question using SparkContext.stop(), or sc.stop().Even then I am getting the following errors. The file is present in the specified folder. 1Cannot load main class from JAR file

Answer 1

To launch a Spark application in cluster mode the general sintax is

$ ./bin/spark-submit --class path.to.your.Class --master yarn --deploy-mode cluster [options] <app jar> [app options]

The easy way in your case would be going under your project directory to launch spark-submit

$ cd /home/username/my_project

under this directory you should have a target directory that is where your app.jar should be

$ ls target/
my_app.jar

As an example of how you should launch your spark-submit

 $ spark-submit \
  --class example.KMeansCoords \
  --name "KMeansCoords" \
  --master yarn-client \
    target/kmeans-1.0.jar \
   /loudacre/devicestatus_etl/*

where

--class <application.MainClass>
--name  <name of your app>
--master <local, yarn, mesos....>
<path to your app.jar>
<options, resources like files to load in your app>

there are many options more, you can follow this link for further details

Running Spark on YARN