How to write a java class with main method to forward to my scala code?

Question

Let me preface by saying I am not very good at programming. Here I have a wordCount Scala code and I want to run as a JAR file for which I need to have a Java class with a main method forwarded to it.

I have created a class but I'm not sure where and how to place it

public class Class{
  object SparkWordCount {
    public static void main(args: Array[String]) {
      val sc = new SparkContext(new SparkConf().setAppName("Spark Count"))
    }
  }
}

I want to create a Java class to my Scala code and run it as a JAR file.

Where is your Scala code? Is there a method in there that you want to call? Or, if it is packaged as a Spark job already, can you run it using the normal tools to submit Spark jobs? — Thilo, Jul 11 '19 at 07:19
This is very unclear what the problem is. I suspect the answer might be to follow some of the getting started with spark and java tutorials such as this one (which I picked randomly from the many Google search results)... https://www.geekmj.org/insights/java-getting-started-725/ — GMc, Jul 11 '19 at 07:20
My scala code is just a wordCount. All I want to do is call it as a java application hence I tried to write it as a Java class — , Jul 11 '19 at 07:23
Can you explain why you want to call it as a *Java* application rather than a *Scala* application? What is the actual problem that you are trying to solve? — Tim, Jul 11 '19 at 07:32
So I want to run it as a JAR file because I need to deploy it in EMR cluster which only takes JAR files. All I need is to write a Java class with main method which forwards the call to my Scala code — , Jul 11 '19 at 07:35
And Scala code can create JAR files. It doesn't sound like you need Java at all. — jwvh, Jul 11 '19 at 08:08
`I want to create a Java class to my Scala code and run it as a JAR file.` what exactly you want to say O_o ? — nomadSK25, Jul 11 '19 at 08:12
ok so if my scala code can be saved and run as a JAR file can you please tell me how to do this? Sorry I know i seem very stupid but I am new to this — , Jul 11 '19 at 08:33
Use the `scalac` compiler to compile the source files to `.class` files. Then create the JAR file the same way you would with Java `.class` files, usually with the `jar` command. — jwvh, Jul 11 '19 at 09:12

Mario Galic · Answer 1 · 2019-07-11T11:41:49.447

Consider using sbt to package your Scala application as a JAR like so

sbt package

However, this will package only your applications's classes, without the dependencies. Package that includes everything is usually called a fat JAR and sbt-assembly is an sbt plugin that enables creation of such packages like so

sbt assembly

To create Apache Spark fat jar try exploring the following answer https://stackoverflow.com/a/28498443/5205022

Try first packaging a simple hello world application before progressing to building fat apache spark jars. For example, install sbt, and then execute the following command

sbt new scala/scala-seed.g8

and then try exploring what happens when you execute sbt package within the created hello world application. The jar package should end up under target directory.

How to write a java class with main method to forward to my scala code?

1 Answers1