3

On my Ubuntu x32, Java x32, Hadoop 2.5.0 I execute:

hduser@master:~$ hdfs dfs -ls /
14/08/19 08:51:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 3 items
drwxr-xr-x   - hduser supergroup          0 2014-08-19 08:37 /data
drwxr-xr-x   - hduser supergroup          0 2014-08-19 08:37 /export
drwxrwx---   - hduser supergroup          0 2014-08-19 08:32 /tmp

Can someone tell me how to fix the warning?

Java version:

hduser@master:~$ java -d64 -version
Error: This Java instance does not support a 64-bit JVM. Please install the desired version.

hduser@master:~$ java -d32 -version
java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) Client VM (build 24.65-b04, mixed mode)

Ubuntu version:

hduser@master:~$ uname -a Linux master 3.13.0-29-generic #53-Ubuntu SMP Wed Jun 4 21:02:19 UTC 2014 i686 i686 i686 GNU/Linux

Hadoop version

hduser@master:~$ hadoop version Hadoop 2.5.0 Subversion http://svn.apache.org/repos/asf/hadoop/common -r 1616291 Compiled by jenkins on 2014-08-06T17:31Z Compiled with protoc 2.5.0 From source with checksum 423dcd5a752eddd8e45ead6fd5ff9a24 This command was run using /home/hduser/hadoop-2.5.0/share/hadoop/common/hadoop-common-2.5.0.jar

EDIT:

.bashrc

export SQOOP_HOME=/home/hduser/sqoop
export JAVA_HOME=$(readlink -f /usr/bin/javac | sed "s:/bin/javac::")
export PATH=$PATH:$SQOOP_HOME/bin
export HADOOP_COMMON_HOME=/home/hduser/hadoop
export HADOOP_MAPRED_HOME=/home/hduser/hadoop
export HADOOP_HOME=/home/hduser/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
export PIG_HOME=/home/hduser/pig
export PATH=$PATH:$PIG_HOME/bin
export HBASE_HOME=/home/hduser/hbase
export HIVE_HOME=/home/hduser/hive
export PATH=$PATH:$HIVE_HOME/bin
Interfector
  • 1,868
  • 1
  • 23
  • 43
  • maybe one of the answers to this post http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-error-on-centos will help you – vefthym Aug 19 '14 at 11:05
  • That link has already been suggested, but it does not solve my problem. – Interfector Aug 19 '14 at 11:58

1 Answers1

2

Set LD_LIBRARY_PATH on your environment pointing to the native library path present in HADOOP_HOME/lib/native

EDIT:

This configuration may vary depending on your Hadoop distribution or linux flavor. Here is a good link with good explanation for CentOS.

Community
  • 1
  • 1
Ankur Shanbhag
  • 7,746
  • 2
  • 28
  • 38
  • What path have you set? Can you share some details of the same – Ankur Shanbhag Aug 19 '14 at 10:06
  • I have appended `export LD_LIBRARY_PATH=$HADOOP_HOME/lib/native` to my `.bashrc` which you have listed in the question. I have then _sourced_ the file and executed `hdfs dfs -ls /` only to get the same warning. I did not have this problem while using Hadoop 2.4.1 and 2.4.0 – Interfector Aug 19 '14 at 10:11
  • I have seen that post, but I could not find a definitive solution. – Interfector Aug 19 '14 at 11:03