1

I am following this hadoop mapreduce tutorial given by Apache. The Java code given there uses these Apache-hadoop classes: 

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

But I could not understand where to download these Jars from. On searching internet for these classes I could see they are available here.

But what is the formal/authentic Apache repository for these and Jars?

If jars are shipped along with hadoop, please let me know the path.

EDIT : Other question does not give clear instructions. I found answer as follows

This tutorial mentions:

Download Hadoop-core-1.2.1.jar, which is used to compile and execute the MapReduce program. Visit the following link http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core/1.2.1 to download the jar.

So this looks authentic repository.

vefthym
  • 7,422
  • 6
  • 32
  • 58
Kaushik Lele
  • 6,439
  • 13
  • 50
  • 76
  • try downloading the hadoop distribution from https://hadoop.apache.org/releases.html and check it's lib folder – DesirePRG Aug 06 '15 at 10:51
  • possible duplicate of [How to import org.apache Java dependencies w/ or w/o Maven](http://stackoverflow.com/questions/11345923/how-to-import-org-apache-java-dependencies-w-or-w-o-maven) – Dan Ciborowski - MSFT Aug 06 '15 at 10:55
  • In this Thread there are answers to the utilization of jar files : https://stackoverflow.com/questions/29551179/hadoop-eclipse-exception-in-thread-main-java-lang-noclassdeffounderror-org/57544077#57544077 – Colonna Maurizio Aug 19 '19 at 10:47

8 Answers8

2

This tutorial mentions :

Download Hadoop-core-1.2.1.jar, which is used to compile and execute the MapReduce program. Visit the following link http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core/1.2.1 to download the jar.

So here you can find all the jars for different versions

Kaushik Lele
  • 6,439
  • 13
  • 50
  • 76
2

The best way is download Hadoop (3.x.y) And include the below jars from hadoop-3.x.y/share/hadoop/mapreduce

1. hadoop-common-3.x.y.jar 2. hadoop-mapreduce-client-core-3.x.y.jar

That worked for me!

Madhusoodan P
  • 681
  • 9
  • 20
0

The tutorial you are following uses Hadoop 1.0. Which means the jars that you have and the ones that the tutorial is using is different. If you are using Hadoop 2.X, follow a tutorial that makes use of exactly that version. You don't need to download jars from a third party, you just need to know the proper use of the API of that specific hadoop version.

Reda
  • 91
  • 1
  • 7
0

Using NetBeans I create a new Maven project.

Then under project files, I open the pom.xml.

I add inside of 

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-core</artifactId>
        <version>0.20.2</version>
    </dependency>

After building with dependencies I am now ready to code.

Dan Ciborowski - MSFT
  • 6,807
  • 10
  • 53
  • 88
0

With current version 2.7.1, I was stumbling at Missing artifact org.apache.hadoop:hadoop-mapreduce:jar:2.7.1, but found out that this jar appears to be split up into various smaller ones.

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>2.7.1</version>
</dependency>

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-mapreduce-client-common</artifactId>
    <version>2.7.1</version>
</dependency>

...worked for me (...no clue what this is meant for: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce/2.7.1/ )

Lost Carrier
  • 123
  • 1
  • 7
0

If you get such type of error then just type the command on terminal:

export HADOOP_HOME=$(hadoop classath)

note:You have to check for your own hadoop configured name in ./bashrc file. At the time of hadoop installation we set the Hadoop and java path in .bashrc file. We have to Check here in below we can see that next to export .

Priyanka Yemul
  • 1
  • 3
0

Try compiling using:
javac -cp $(hadoop classpath) MapRTest.java.
In most cases, the files are already present with the downloaded hadoop. For more info, look into this.

subtleseeker
  • 4,415
  • 5
  • 29
  • 41
0

javac -cp /usr/hdp/2.6.2.0-205/hadoop-mapreduce/:/usr/hdp/2.6.2.0-205/hadoop/:. MyTest.java

Worked for me in CloudxLab.

San4musa
  • 277
  • 2
  • 12