1

My issue is similar to this one: How do I build/run this simple Mahout program without getting exceptions?, but a bit more complicated.

I'm writing Hadoop code, and would like to use Mahout math libs (e.g. SparseRowMatrix & VectorWritable classes).

The problem is that the former example class lies at math folder of latest SNAPSHOT of Mahout, while the latter is placed under core folder, and both share the same path as of the point after the root folder (org.apache.mahout.math).

The problem is that I have found no way to make Hadoop jar file to import properly the classes that concern math stuff, but lie under the Core folder of Mahout project.

I'm working with Eclipse and to run successfully my jobs, I have ,each time I export a new jar, to search down the classpath tree of core folder & bundle the respective math folder into my project.

How can I avoid this procedure?

Usama Abdulrehman
  • 1,041
  • 3
  • 11
  • 21
user1519128
  • 131
  • 2
  • 7

1 Answers1

0

Are you using Mahout code directly in Eclipse project, or you're using other build tool? I would recommend to use maven-based project, and declare dependency on needed Mahout version in it. Plus you'll get ability to create striped job jar, like this was done for examples from book 'Mahout in Action' - look onto pom.xml there...

Maven support in eclipse is implemented through m2eclipse plugin...

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • I used this link : http://domengrabec.wordpress.com/2010/12/14/installing-maven-on-eclipse/, to import finally mahout as an existing maven project. The thing is that when I tried to use maven dependency management to my job through eclipse, I messed it up a bit. Could you please give more details on what exactly to do? Thanks for answering! – user1519128 Aug 01 '12 at 10:31
  • 1
    I wrote blog post about working with MiA examples, but I think, that it could be also applicable to your case: http://alexott.blogspot.de/2012/07/getting-started-with-examples-from.html – Alex Ott Aug 01 '12 at 13:31
  • Thank you very much for your responses, but unfortunately I wasn't able to solve the issue.. It's like the hadoop job cannot retrieve the "org.apache.hadoop.math" folder placed under "core" dir of mahout, unless I bundle it manually into my exported jar. Perhaps, is there a way of making eclipse always export a specific directory of another project, as a configuration of my project? Because the whole problem is that I have each time I export my jar, to go down the dependency tree of mahout's core project and find the required dependency folder. Thanks again for your time :-) – user1519128 Aug 02 '12 at 00:45
  • I can't say much about Eclipse way of solving such problem - I always use Maven that packs everything for me, independent on IDE used... – Alex Ott Aug 02 '12 at 06:19