3

I am working with Apache Spark through Maven, and I am trying to modify the source by including a 3rd party jar and trying to utilize some methods within it.

I get the following error when compiling the Spark project using

mvn -Dhadoop.version=2.2.0 -Dscala-2.11 -DskipTests clean package

not found: object edu
[ERROR] import edu.xxx.cs.aggr._

I modified ResultTask.scala to contain an import statement. So, maven is unable to find the jar I am trying to use and link it with the project.

I have added a dependency to the pom.xml file such as this: 

<dependency>
    <groupId>edu.xxx.cs</groupId>
    <artifactId>aggr</artifactId>
    <version>0.99</version>
    <scope>system</scope>
    <systemPath>${basedir}/aggr.jar</systemPath>
</dependency>

The jar file I am trying to link is located in the same directory as the spark pom.xml file. I added this dependency to pom.xml. I inserted it in between 2 existing dependencies within the pom.xml file. I'm not sure whats wrong, but I would just like the jar to get linked for now, so that I can use the methods within it. I'm also not sure if I should be using anything specific for the groupId, artifactId, and version. edu.xxx.cs.aggr is the root package which contains other source files and packages. I would appreciate any help.

UPDATE

I used 

mvn install:install-file -Dfile=<path-to-file> -DgroupId=edu.xxx.cs -DartifactId=aggr -Dversion=0.99 -Dpackaging=jar to install the jar to the .m2 repo. I checked the repo to see if it was installed, and it was.

I changed the dependency in pom.xml to

<dependency>
        <groupId>edu.purdue.cs</groupId>
        <artifactId>aggr</artifactId>
        <version>0.99</version>
</dependency>

I still get the same error.

RagHaven
  • 4,156
  • 21
  • 72
  • 113
  • Try manually adding it to your local maven repository in location /edu/xxx/cs/aggr/0.99/aggr-0.99.jar – Davio May 20 '15 at 10:47
  • That's a wrong way and there is no guarantee it will work always. Maven provide its native solutions instead of changing its artifacts base manually. – nickolay.laptev May 20 '15 at 11:00
  • If you open up that aggr.jar file in a zip tool, IS there actually a directory 'edu' in there I wonder? – Gimby May 20 '15 at 11:14
  • I've used this jar in a simple java project that I have created on eclipse. The import statement edu.xxx.cs.aggr doesn't produce any errors in that project. – RagHaven May 20 '15 at 11:16

4 Answers4

7

this is how I add system dependency to my maven pom.xml. with in the project root path, I have created lib directory and there I have placed my jar file.

<dependency>
    <groupId>com.sshx</groupId>
    <artifactId>sshx</artifactId>
    <version>1.0</version>
    <scope>system</scope>
    <systemPath>${basedir}/lib/sshxcute-1.0.jar</systemPath>
</dependency>

if still you face the same issue try adding the dependency manually by issuing the following command from the jar file location

mvn install:install-file -Dfile=sshxcute-1.0.jar -DgroupId=com.sshx -DartifactId=sshx -Dversion=1.0 -Dpackaging=jar

this command will add the jar to your .m2 repository as a dependency and you need to change the pom.xml dependency as follows:

<dependency>
    <groupId>com.sshx</groupId>
    <artifactId>sshx</artifactId>
    <version>1.0</version>
</dependency>

once you are done, issue mvn clean install command from command prompt and build your application.

However, another option is to create a local repository. See at this thread: How to include local jar files in Maven project

Ram Ghadiyaram
  • 28,239
  • 13
  • 95
  • 121
Prasad Khode
  • 6,602
  • 11
  • 44
  • 59
  • I manually added the jar file to my .m2 repo. Now, it is under the proper directories and the jar is there. But, I still get this error – RagHaven May 20 '15 at 10:57
  • does manually means issuing the **mvn install** command?? have you updated your dependency in your pom.xml file as mentioned above? – Prasad Khode May 20 '15 at 10:58
  • I used mvn install:install-file -Dfile= -DgroupId=edu.xxx.cs -DartifactId=aggr -Dversion=0.99 -Dpackaging=jar I changed the dependency to edu.xxx.cs aggr 0.99 – RagHaven May 20 '15 at 11:06
  • try to refresh your project once from IDE or issue **mvn clean install** command and try – Prasad Khode May 20 '15 at 11:08
  • I don't have this setup on an IDE. I am making changes via sublime and building using mvn. So, I should just use mvn clean install from the dir that contains pom.xml? – RagHaven May 20 '15 at 11:10
  • yes, you need to use mvn clean install to re-build the project so that your changes take effect – Prasad Khode May 20 '15 at 11:11
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/78307/discussion-between-androiddev93-and-prasad-khode). – RagHaven May 20 '15 at 11:12
  • Simplest solution is to start using a repository manager and install that artifact there. Afterwards use it as any other dependency without system scope. – khmarbaise May 20 '15 at 12:58
0

Try to include the filename into the variable. For what I know you may not mix variables and paths within the systemPath parameter.

CruLPlay
  • 61
  • 3
0

There must be several options but the only way. The way depends on how Maven really works. It works with dependency repositories. So to compile successfully you need to specify valid repository for you JAR. You can use your local repository for this. To add JAR there you can invoke "mvn install" command for that JAR. As an alternative you can specify in your POM file the location of local repository to use for Maven.

nickolay.laptev
  • 2,253
  • 1
  • 21
  • 31
0

You can install your Jar to local maven repository by using following command:

mvn install:install-file -Dfile=<path-to-file> -DgroupId=<group-id> -DartifactId=<artifact-id> -Dversion=<version> -Dpackaging=<packaging>
Sachin Gupta
  • 7,805
  • 4
  • 30
  • 45
  • I manually added the jar to the local repo. I created the appropriate directory structure /edu/xxx/cs/aggr/0.99/aggr-0.99.jar. I compiled again and I still got the same error – RagHaven May 20 '15 at 10:56
  • don't try to create it manually. Just use command it will create it for you automatically. I think this command will create some 2-3 supporting files with the jar. – Sachin Gupta May 20 '15 at 10:57
  • Thats what I did. I used mvn install:install-file -Dfile= -DgroupId=edu.xxx.cs -DartifactId=aggr -Dversion=0.99 -Dpackaging=jar – RagHaven May 20 '15 at 11:04
  • after installing the jar don't forget to refresh the project. or try `mvn clean` – Sachin Gupta May 20 '15 at 11:09
  • when I build the project I already use clean by doing mvn -Dhadoop.version=2.2.0 -Dscala-2.11 -DskipTests clean package. I tried mvn clean before building it, I still get the error. – RagHaven May 20 '15 at 11:11