0

I am migrating a map-reduce java-project (built using maven) from Horton Works to Big Top.

I am trying to figure out what is the best way to ensure that the depedency versions in my java-project match the jar files deployed on the cluster by Big Top.

We are currently targetting Big Top 3.2.0.

I am inspecting their BOM file and using those versions in my pom file.

For example, when we were using hdp I had something like

   <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.11</artifactId>
        <version>2.3.2.3.1.4.0-315</version>          
    </dependency>

According to the Big Top BOM file the Spark Version is 3.2.3 & Scala Library Version is 2.12.13. Does that mean that the new maven depdency in our project pom file should be

   <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.12</artifactId>
        <version>3.2.3</version>          
    </dependency>

Is there a place where the exact maven dependencies are listed? Is this the correct way to migrate our project's POM file?

hba
  • 7,406
  • 10
  • 63
  • 105

0 Answers0