$SPARK_HOME/bin/spark-shell --packages datastax:spark-cassandra-connector:2.0.0-M2-s_2.11 --conf spark.cassandra.connection.host=127.0.0.1
Ivy Default Cache set to: /home/gopalkrishna/.ivy2/cache
The jars for the packages stored in: /home/gopalkrishna/.ivy2/jars
:: loading settings :: url = jar:file:/home/gopalkrishna/INSTALL/spark-2.1.0-bin-hadoop2.6/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
datastax#spark-cassandra-connector added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
confs: [default]
found datastax#spark-cassandra-connector;2.0.0-M2-s_2.11 in spark-packages
downloading http://dl.bintray.com/spark-packages/maven/datastax/spark-cassandra-connector/2.0.0-M2-s_2.11/spark-cassandra-connector-2.0.0-M2-s_2.11.jar ...
[SUCCESSFUL ] datastax#spark-cassandra-connector;2.0.0-M2-s_2.11!spark-cassandra-connector.jar (7998ms)
:: resolution report :: resolve 33340ms :: artifacts dl 8002ms
:: modules in use:
datastax#spark-cassandra-connector;2.0.0-M2-s_2.11 from spark-packages in [default]
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 7 | 1 | 1 | 0 || 1 | 1 |
---------------------------------------------------------------------
:: problems summary ::
:::: WARNINGS
module not found: commons-beanutils#commons-beanutils;1.8.0
==== local-m2-cache: tried
file:/home/gopalkrishna/.m2/repository/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.pom
-- artifact commons-beanutils#commons-beanutils;1.8.0!commons-beanutils.jar:
file:/home/gopalkrishna/.m2/repository/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.jar
==== local-ivy-cache: tried
/home/gopalkrishna/.ivy2/local/commons-beanutils/commons-beanutils/1.8.0/ivys/ivy.xml
-- artifact commons-beanutils#commons-beanutils;1.8.0!commons-beanutils.jar:
/home/gopalkrishna/.ivy2/local/commons-beanutils/commons-beanutils/1.8.0/jars/commons-beanutils.jar
==== central: tried
https://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.pom
-- artifact commons-beanutils#commons-beanutils;1.8.0!commons-beanutils.jar:
https://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.pom
-- artifact commons-beanutils#commons-beanutils;1.8.0!commons-beanutils.jar:
http://dl.bintray.com/spark-packages/maven/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.jar
module not found: org.joda#joda-convert;1.2
==== local-m2-cache: tried
file:/home/gopalkrishna/.m2/repository/org/joda/joda-convert/1.2/joda-convert-1.2.pom
-- artifact org.joda#joda-convert;1.2!joda-convert.jar:
file:/home/gopalkrishna/.m2/repository/org/joda/joda-convert/1.2/joda-convert-1.2.jar
==== local-ivy-cache: tried
/home/gopalkrishna/.ivy2/local/org.joda/joda-convert/1.2/ivys/ivy.xml
-- artifact org.joda#joda-convert;1.2!joda-convert.jar:
/home/gopalkrishna/.ivy2/local/org.joda/joda-convert/1.2/jars/joda-convert.jar
==== central: tried
https://repo1.maven.org/maven2/org/joda/joda-convert/1.2/joda-convert-1.2.pom
-- artifact org.joda#joda-convert;1.2!joda-convert.jar:
https://repo1.maven.org/maven2/org/joda/joda-convert/1.2/joda-convert-1.2.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/org/joda/joda-convert/1.2/joda-convert-1.2.pom
-- artifact org.joda#joda-convert;1.2!joda-convert.jar:
http://dl.bintray.com/spark-packages/maven/org/joda/joda-convert/1.2/joda-convert-1.2.jar
module not found: joda-time#joda-time;2.3
==== local-m2-cache: tried
file:/home/gopalkrishna/.m2/repository/joda-time/joda-time/2.3/joda-time-2.3.pom
-- artifact joda-time#joda-time;2.3!joda-time.jar:
file:/home/gopalkrishna/.m2/repository/joda-time/joda-time/2.3/joda-time-2.3.jar
==== local-ivy-cache: tried
/home/gopalkrishna/.ivy2/local/joda-time/joda-time/2.3/ivys/ivy.xml
-- artifact joda-time#joda-time;2.3!joda-time.jar:
/home/gopalkrishna/.ivy2/local/joda-time/joda-time/2.3/jars/joda-time.jar
==== central: tried
https://repo1.maven.org/maven2/joda-time/joda-time/2.3/joda-time-2.3.pom
-- artifact joda-time#joda-time;2.3!joda-time.jar:
https://repo1.maven.org/maven2/joda-time/joda-time/2.3/joda-time-2.3.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/joda-time/joda-time/2.3/joda-time-2.3.pom
-- artifact joda-time#joda-time;2.3!joda-time.jar:
http://dl.bintray.com/spark-packages/maven/joda-time/joda-time/2.3/joda-time-2.3.jar
module not found: io.netty#netty-all;4.0.33.Final
==== local-m2-cache: tried
file:/home/gopalkrishna/.m2/repository/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.pom
-- artifact io.netty#netty-all;4.0.33.Final!netty-all.jar:
file:/home/gopalkrishna/.m2/repository/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.jar
==== local-ivy-cache: tried
/home/gopalkrishna/.ivy2/local/io.netty/netty-all/4.0.33.Final/ivys/ivy.xml
-- artifact io.netty#netty-all;4.0.33.Final!netty-all.jar:
/home/gopalkrishna/.ivy2/local/io.netty/netty-all/4.0.33.Final/jars/netty-all.jar
==== central: tried
https://repo1.maven.org/maven2/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.pom
-- artifact io.netty#netty-all;4.0.33.Final!netty-all.jar:
https://repo1.maven.org/maven2/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.pom
-- artifact io.netty#netty-all;4.0.33.Final!netty-all.jar:
http://dl.bintray.com/spark-packages/maven/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.jar
module not found: com.twitter#jsr166e;1.1.0
==== local-m2-cache: tried
file:/home/gopalkrishna/.m2/repository/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.pom
-- artifact com.twitter#jsr166e;1.1.0!jsr166e.jar:
file:/home/gopalkrishna/.m2/repository/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.jar
==== local-ivy-cache: tried
/home/gopalkrishna/.ivy2/local/com.twitter/jsr166e/1.1.0/ivys/ivy.xml
-- artifact com.twitter#jsr166e;1.1.0!jsr166e.jar:
/home/gopalkrishna/.ivy2/local/com.twitter/jsr166e/1.1.0/jars/jsr166e.jar
==== central: tried
https://repo1.maven.org/maven2/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.pom
-- artifact com.twitter#jsr166e;1.1.0!jsr166e.jar:
https://repo1.maven.org/maven2/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.pom
-- artifact com.twitter#jsr166e;1.1.0!jsr166e.jar:
http://dl.bintray.com/spark-packages/maven/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.jar
module not found: org.scala-lang#scala-reflect;2.11.8
==== local-m2-cache: tried
file:/home/gopalkrishna/.m2/repository/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.pom
-- artifact org.scala-lang#scala-reflect;2.11.8!scala-reflect.jar:
file:/home/gopalkrishna/.m2/repository/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar
==== local-ivy-cache: tried
/home/gopalkrishna/.ivy2/local/org.scala-lang/scala-reflect/2.11.8/ivys/ivy.xml
-- artifact org.scala-lang#scala-reflect;2.11.8!scala-reflect.jar:
/home/gopalkrishna/.ivy2/local/org.scala-lang/scala-reflect/2.11.8/jars/scala-reflect.jar
==== central: tried
https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.pom
-- artifact org.scala-lang#scala-reflect;2.11.8!scala-reflect.jar:
https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar
==== spark-packages: tried
http://dl.bintray.com/spark-packages/maven/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.pom
-- artifact org.scala-lang#scala-reflect;2.11.8!scala-reflect.jar:
http://dl.bintray.com/spark-packages/maven/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar
::::::::::::::::::::::::::::::::::::::::::::::
:: UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: commons-beanutils#commons-beanutils;1.8.0: not found
:: org.joda#joda-convert;1.2: not found
:: joda-time#joda-time;2.3: not found
:: io.netty#netty-all;4.0.33.Final: not found
:: com.twitter#jsr166e;1.1.0: not found
:: org.scala-lang#scala-reflect;2.11.8: not found
::::::::::::::::::::::::::::::::::::::::::::::
:::: ERRORS
Server access error at url https://repo1.maven.org/maven2/datastax/spark-cassandra-connector/2.0.0-M2-s_2.11/spark-cassandra-connector-2.0.0-M2-s_2.11.pom (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/datastax/spark-cassandra-connector/2.0.0-M2-s_2.11/spark-cassandra-connector-2.0.0-M2-s_2.11.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/datastax/spark-cassandra-connector/2.0.0-M2-s_2.11/spark-cassandra-connector-2.0.0-M2-s_2.11-sources.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/datastax/spark-cassandra-connector/2.0.0-M2-s_2.11/spark-cassandra-connector-2.0.0-M2-s_2.11-src.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/datastax/spark-cassandra-connector/2.0.0-M2-s_2.11/spark-cassandra-connector-2.0.0-M2-s_2.11-javadoc.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.pom (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/org/joda/joda-convert/1.2/joda-convert-1.2.pom (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/org/joda/joda-convert/1.2/joda-convert-1.2.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/joda-time/joda-time/2.3/joda-time-2.3.pom (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/joda-time/joda-time/2.3/joda-time-2.3.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.pom (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.pom (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/com/twitter/jsr166e/1.1.0/jsr166e-1.1.0.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.pom (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
Server access error at url https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.8/scala-reflect-2.11.8.jar (javax.net.ssl.SSLException: Received fatal alert: protocol_version)
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: commons-beanutils#commons-beanutils;1.8.0: not found, unresolved dependency: org.joda#joda-convert;1.2: not found, unresolved dependency: joda-time#joda-time;2.3: not found, unresolved dependency: io.netty#netty-all;4.0.33.Final: not found, unresolved dependency: com.twitter#jsr166e;1.1.0: not found, unresolved dependency: org.scala-lang#scala-reflect;2.11.8: not found]
at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1078)
at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:296)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:160)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Asked
Active
Viewed 853 times
0
Alex Ott
- 80,552
- 8
- 87
- 132
Venu Gopal
- 11
- 2
-
Welcome to Stack Overflow! I strongly recommend you have a look at [How do I ask a good question?](https://stackoverflow.com/help/how-to-ask) Dumping code and output with no additional information is both unclear and not useful to others using the site. It helps if you start by explaining what you want, and detailing whatever research or other efforts you've tried. – B. Shefter Feb 28 '20 at 19:04
-
Does this answer your question? [spark shell dependency exception](https://stackoverflow.com/questions/51409064/spark-shell-dependency-exception) – Alex Ott Feb 29 '20 at 08:59
-
it's a problem with Java TLS version - see linked answer – Alex Ott Feb 29 '20 at 08:59
1 Answers
1
But why are you using the version 2.0.0-M2? It's 3.5 years old, and is a preview of the new functionality, that isn't supposed to be used in production...
You need to use one of the latest versions, as pointed in the connector documentation, like:
bin/spark-shell --packages com.datastax.spark:spark-cassandra-connector_2.11:2.4.2
P.S. there are sometimes problems with Spark packages site, so you can try 2.4.0 instead of 2.4.2
Alex Ott
- 80,552
- 8
- 87
- 132
-
-
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: com.datastax.spark#spark-cassandra-connector_2.11;2.4.2: not found] – Venu Gopal Mar 01 '20 at 02:55
-
Can you try with 2.4.0 - sometimes there are problems with Spark packages site – Alex Ott Mar 01 '20 at 08:18