30

I am running into an issue with my Maven POM file where its unable to find spark dependency and is returning with error: Failed to read artifact descriptor for org.apache.spark:spark-streaming-kafka_2.10:jar:1.2.1

I've confirmed its not an issue with any corporate firewall as every other dependency is loaded in properly, just this one.

I have also been able to confirm in my maven settings it is attempting to pull from the following repo. I tried deleting the .m2 repo on my local machine in order to reload it, still no dice.

http://repo.maven.apache.org/maven2/org/apache/spark/spark-streaming-kafka_2.10/1.2.1/

Attached below is my pom file

<groupId>my.group.id</groupId>
<artifactId>sentiment</artifactId>
<version>1.0-SNAPSHOT</version>
<name>NPITWITTER</name>

<properties>
</properties>

    <dependencies>
        <dependency>
            <groupId>com.sparkjava</groupId>
            <artifactId>spark-core</artifactId>
            <version>1.1.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>


    </dependencies>

user2100493
  • 1,258
  • 4
  • 15
  • 26
  • 1
    This dependency correctly exists: http://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka_2.10/1.2.1 so the problem is elsewhere. – Tunaki Jan 30 '16 at 18:54

8 Answers8

117

In Intellij, I had the same issue,

I did the proxy settings also, and ran the command mvn –U clean install but it's not working.

I got its resolved with the below setting enabled (highlighted in yellow)

In Intellij, go to File -> Settings -> Build, Execution, Deployments -> Maven Check the box, ‘Always update snapshots

Intellij Settings window snapshot

Youcef LAIDANI
  • 55,661
  • 15
  • 90
  • 140
Raju
  • 1,179
  • 2
  • 7
  • 3
17

Resolved it, was due to a proxy setting that was enabled. Re-ran with mvn -U clean install and all was repaired.

einverne
  • 6,454
  • 6
  • 45
  • 91
user2100493
  • 1,258
  • 4
  • 15
  • 26
6

Following worked for IDEA 17,

  • Go to File
  • Select Settings
  • Select Build, Execution, Deployments
  • Select Build Tools from drop down
  • Select Maven from drop down
  • Tick the Always update snapshots check box
prime
  • 14,464
  • 14
  • 99
  • 131
6

I guess IntelliJ comes with and ambedded Maven installation, so just make sure your IDE is pointing to the right Maven home directory: and User settings file: values, these are under:

  • Go to File
  • Select Settings
  • Select Build, Execution, Deployments
  • Select Build Tools from drop down
  • Select Maven from drop down
Sandoval0992
  • 1,307
  • 3
  • 18
  • 24
2

I had just solved the issue. Please, go in the Maven tab of the Build, Execution, Deployments and set it to the settings provided.

enter image description here

Arefe
  • 11,321
  • 18
  • 114
  • 168
1

The above solution had to extend further to resolve my issue(In IntelliJ idea).

Step 1

Go to File -> Settings -> (Build, Execution, Deployment) -> Maven and tick the Always update snapshot option and then click on Apply -> Ok

enter image description here

Step 2

Go to File -> Invalidate Cache/Restart -> Invalidate and Restart

enter image description here

Step 3

Build the project after step 2 and see whether the error is fixed. If not, Restart the computer and then run the project.

I had to follow all the 3 steps to get the error fixed

Niroshan Ratnayake
  • 3,433
  • 3
  • 20
  • 18
0

In my case, it was proxy related problem. I am highlighting few of areas where you may need to verify and correct the same.

  1. Intellij proxy settings from preference window
  2. maven settings.xml entry from your ~/.m2/settings.xml or /usr/local/Cellar/maven//libexec/conf/settings.xml
  3. System proxy

Hope this will help.

Dhrubo
  • 705
  • 1
  • 14
  • 33
-1

Go to file => Settings => Build, Execution, Deployment => Maven => Maven home directory: change from Bundled (Maven 3) to Bundled (Maven 2)