7

I've installed Hadoop 2.7.2 single node on Ubuntu and I want to run the java wordcount program. The compilation and the creation of the jar file are done succesfully, but when I run the jar file on Hadoop I receive this message:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

I set environment variables by editing .bashrc file:

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib

When I type hadoop checknative -a I get this:

hadoop: true /usr/local/hadoop/lib/native/libhadoop.so.1.0.0
zlib: true /lib/x86_64-linux-gnu/libz.so.1
snappy: false 
lz4: true revision:99
bzip2: false
openssl: true /usr/lib/x86_64-linux-gnu/libcrypto.so

16/05/09 00:48:53 INFO util.ExitUtil: Exiting with status 1

Hadoop version: 2.7.2

Ubuntu version: 14.04

Could anyone give some clues about the issue?

El Mehdi Belgasmi
  • 71
  • 1
  • 1
  • 4
  • 3
    Possible duplicate of [Hadoop "Unable to load native-hadoop library for your platform" warning](http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning) – 32cupo Jan 26 '17 at 11:22

5 Answers5

4

Move your compiled native library files to $HADOOP_HOME/lib folder.

Then set your environment variables by editing .bashrc file

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib  
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib"

Make sure your compiled native library files are in $HADOOP_HOME/lib folder.

It should fix the issue.

Nishu Tayal
  • 20,106
  • 8
  • 49
  • 101
1

Add the commands lines below to hadoop-env.sh, it should suppress the errors encountered

export HADOOP_HOME_WARN_SUPPRESS=1
export HADOOP_ROOT_LOGGER="WARN,DRFA"
Praveen
  • 1,449
  • 16
  • 25
Mike Onuorah
  • 11
  • 1
  • 1
    I configured hadoop on raspberry where none of the library files are native and while it all works it's nice to suppress the warnings. Many Thanks –  Mar 16 '19 at 21:10
0

Try to load the hadoop jars-"hadoop-common.jar and hadoop-core.jar" to your class path. You can simply do it in eclipse ans while creating a jar file those jars will be referenced automatically.

Ishan Kumar
  • 1,941
  • 3
  • 20
  • 29
0

I had this error.

I solved it by updating the winutils.exe and hadoop.dll files to the later version. :)

https://github.com/kontext-tech/winutils/tree/master/hadoop-3.3.1 (where I was able to find different versions)

user22276415
  • 1
  • As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – hossein Jul 27 '23 at 10:57
-3

edit the file hadoop-env.sh in /usr/local/etc/hadoop

Adding Hadoop library into LD_LIBRARY_PATH :

export LD_LIBRARY_PATH=/usr/local/hadoop/lib/native/:$LD_LIBRARY_PATH

try it, works for me

MatejMecka
  • 1,448
  • 2
  • 24
  • 37
strom
  • 7
  • 4