Where are hadoop jar files in hadoop 2?

Question

I am trying to implement one sample word count program using Hadoop. I have downloaded and installed Hadoop 2.0.0. I want to do this sample program using Eclipse because i think later in my real project I have to use Eclipse only.

I am not able to find Hadoop related jar files like hadoop-core.jar and other required jar files. I searched in all the folders of 2.0 hadoop but couldn't find those files. Those same files are available in 1.0 version of Hadoop but not in 2.0 version. I would like to know where can I get these files?

I am not able to find much information about 2.0 version.

please help

aa8y · Accepted Answer · 2015-02-03T18:24:56.443

37

Actually, I faced a similar issue in the beginning. Apparently, there is no hadoop-core.jar for version 2. You'll have to download the Hadoop Common and MapReduce Client Core jars and use them both. I am sure it'll solve your problems. Hope this helps!

Edit: I just updated the links such that people can choose the appropriate version from the repository since we now come way past the initial version 2.

edited Feb 03 '15 at 18:24

answered Mar 04 '13 at 12:54

aa8y

3,854
4
37
62

1

What about dependent libraries? It causes runtime errors later: http://stackoverflow.com/questions/20763553/hadoop-2-0-jar-files. – Igor Ševo Dec 24 '13 at 17:08
1

@IgorSevo I think you might need to add some dependent libraries, but the question did not ask for it. Also, if you use `Gradle` for dependency management, it automatically includes the dependent libraries and the code works just fine with the above two jars. – aa8y Oct 22 '14 at 16:34
as mentioned in the below reply. They will be available at `${HADOOP_HOME}/share/hadoop/common` and `${HADOOP_HOME}/share/hadoop/mapreduce` – Jagadish Talluri Nov 19 '15 at 15:00

score 12 · Answer 2 · edited Feb 17 '17 at 21:36

12

Including JARs in folders listed below worked for me:

share\hadoop\common       
share\hadoop\common\lib   
share\hadoop\mapreduce    
share\hadoop\mapreduce\lib
share\hadoop\yarn         
share\hadoop\yarn\lib

edited Feb 17 '17 at 21:36

OneCricketeer

179,855
19
132
245

answered May 24 '14 at 13:14

Anonymous

123
1
4

The slashes have to be forward!! – nishant Sep 21 '17 at 05:44

Chong Tang · Answer 3 · 2015-10-24T14:42:10.157

6

hadoop-core-${HADOOP_VERSION}.jar is in

${HADOOP_HOME}/share/hadoop/common

directory, not ${HADOOP_HOME} directory.

edited Oct 24 '15 at 14:42

answered Feb 04 '15 at 00:11

Chong Tang

2,066
15
12

${HADOOP_HOME}/share/hadoop/common – Jignesh Rawal Oct 23 '15 at 10:19
simple and good answer. Upvoted. – Jagadish Talluri Nov 19 '15 at 14:55

score 3 · Answer 4 · answered Mar 18 '15 at 05:14

3

This works for me on Mac on Hadoop 2.6.0:

hadoop/2.6.0/libexec/share/hadoop/common/hadoop-common-2.6.0.jar

or just search ".jar" in the hadoop directory, you can see all the .jar files.

answered Mar 18 '15 at 05:14

Yimihua

446
3
6

score 2 · Answer 5 · edited Apr 19 '17 at 07:38

2

hadoop-core-{version}.jar became hadoop-common-{version}.jar starting with Hadoop 2+

edited Apr 19 '17 at 07:38

goto

7,908
10
48
58

answered Feb 16 '16 at 17:51

Venkatt Guhesan

105
4

score 1 · Answer 6 · answered Sep 13 '17 at 18:21

1

We can get the jar files from https://mvnrepository.com website.

Example - https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core/1.2.1

answered Sep 13 '17 at 18:21

Satyajit Mishra

53
5

score 0 · Answer 7 · answered Jul 20 '17 at 06:27

0

On hadoop 2.7.2, I had to set it to the same as HADOOP_COMMON_HOME.

answered Jul 20 '17 at 06:27

Henry

2,870
1
25
17

score -1 · Answer 8 · answered May 18 '16 at 14:39

-1

At below location you can find folder named by distribution which you have installed , inside that there is a jar folder. /opt/cloudera/parcels/[cloudera distribution]/jars/

Example : /opt/cloudera/parcels/CDH-5.4.10/jars

answered May 18 '16 at 14:39

Ankush

11
2

Where are hadoop jar files in hadoop 2?

8 Answers8

Linked