SBT: SPARK2.0.*: SCALA: unresolved dependencies org.glassfish.hk2 and javax.validation

Question

Unable to resolve dependencies while trying to use latest Apache Spark 2.0.2 and Scala 2.11.7 together

I need to use the latest version of spark since it has support for SparkSession. Below configuration working fine for spark 1.6.2. But I want for spark 2.0.*

This is my dependency configuration

scalaVersion := "2.11.7"

val sparkVersion = "2.0.2"

libraryDependencies ++= Seq(
  ws   ,
  "org.apache.spark" %% "spark-core" % sparkVersion,
  "org.apache.spark" %% "spark-sql" % sparkVersion,
  "org.apache.spark" %% "spark-streaming" % sparkVersion,
  "org.mongodb.spark" % "mongo-spark-connector_2.11" % "2.0.0",
  specs2 % Test
)

And this is error message:

Error:Error while importing SBT project:
[info] Resolving org.ow2.asm#asm-tree;4.1 ...
[info] Resolving org.ow2.asm#asm-analysis;4.1 ...
[info] Resolving org.ow2.asm#asm-util;4.1 ...
[info] Resolving jline#jline;2.12.1 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.glassfish.hk2#hk2-utils;2.22.2: not found
[warn]  :: org.glassfish.hk2#hk2-locator;2.22.2: not found
[warn]  :: javax.validation#validation-api;${javax.validation.version}: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.glassfish.hk2#hk2-utils;2.22.2: not found
[error] unresolved dependency: org.glassfish.hk2#hk2-locator;2.22.2: not found
[error] unresolved dependency: javax.validation#validation-api;${javax.validation.version}: not found
[error] Total time: 81 s, completed 3 Dec, 2016 8:51:18 PM

Correct me if anything wrong in applying dependencies. Thank you

Please consider commenting your downvote? – Jyothi Babu Araja Jan 08 '17 at 07:51 — Jyothi Babu Araja, Jan 08 '17 at 07:51

The_Tourist · Answer 1 · 2016-12-03T18:19:23.463

2

There's probably an error in the dependency publication chain. One option would be to exclude the broken dependencies - add exclude("org.glassfish.hk2", "hk2-utils") exclude("org.glassfish.hk2", "hk2-locator") exclude("javax.validation", "validation-api") to whatever Spark dependency that's bringing these in - and explicitly add the working versions of those three dependencies.

Also, I don't see version 2.22.2 for org.glassfish.hk2#hk2-utils or org.glassfish.hk2#hk2-locator.

edited Dec 03 '16 at 18:19

answered Dec 03 '16 at 18:13

The_Tourist

2,048
17
21

How can i report this MavenCentral? Not only these, there are a few more broken dependencies which occur in runtime like `fastxml-jackson` – Jyothi Babu Araja Dec 04 '16 at 11:03

score 0 · Answer 2 · answered Sep 21 '21 at 17:44

Excluding at the org level (as suggested in How to exclude commons logging dependency of spring with ivy? ) helped me.

I had to use something like this :

<dependency org="org.glassfish.jersey.core" name="jersey-client" rev="2.3" conf="binaries->default">
    <exclude  org="org.glassfish.hk2"/>
</dependency>
<dependency org="org.glassfish.hk2" name="hk2-utils" rev="2.2.0-b14" conf="binaries->default"/>
<dependency org="org.glassfish.hk2" name="hk2-locator" rev="2.2.0-b14" conf="binaries->default"/>
<dependency org="org.glassfish.hk2" name="hk2-api" rev="2.2.0-b14" conf="binaries->default"/>

SBT: SPARK2.0.*: SCALA: unresolved dependencies org.glassfish.hk2 and javax.validation

2 Answers2