10

While running hadoop on Windows 7 64-bit, I got stuck on this:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
WARN common.Util: Path /hadoop-2.6.5/data/namenode should be specified as a URI in configuration files. Please update hdfs configuration.
WARN common.Util: Path /hadoop-2.6.5/data/namenode should be specified as a URI in configuration files

How can I solve these warnings?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
inos
  • 101
  • 1
  • 1
  • 3
  • 1
    Possible duplicate: http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning – philantrovert Apr 28 '17 at 07:08
  • Possible duplicate of [Hadoop "Unable to load native-hadoop library for your platform" warning](http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning) –  Apr 28 '17 at 11:00
  • 6
    I believe OP is running this on Windows 7. The links above all point to solutions in CentOS, Ubuntu, etc. – Bookamp Jun 08 '17 at 03:33
  • 2
    I agree with Bookamp. All above links are non-windows specific. I am also facing same issue. I am not trying to building from hadoop source. I downloaded hadoop binaries (2.8.1 tar gz) and trying to run on windows 7 (64 bit) using cygwin but facing same error. Seems IO Native library is an issue here. Can I really not run from binaries ? I don't want to build from source as that has so many other SDKs dependencies. – nirmalsingh Jul 27 '17 at 16:04

2 Answers2

11

There are no native hadoop libraries for Windows (see Native Libraries Guide so the warning is expected and it's just a warning, not an error. The same applies for Mac OS X environments, no native libraries provided either.

If you want to suppress the WARN just edit the $HADOOP_CONF_DIR/log4j.properties (usually something like etc/hadoop/log4j.properties) and add the following: 

log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR

Only errors (and not warnings) will be logged from now on.

RubenLaguna
  • 21,435
  • 13
  • 113
  • 151
  • Have you read this Hadoop on Linux includes optional Native IO support. However Native IO is mandatory on Windows and without it you will not be able to get your installation working. You must follow all the instructions from BUILDING.txt to ensure that Native IO support is built correctly. from here https://wiki.apache.org/hadoop/Hadoop2OnWindows – Dims Apr 22 '18 at 17:51
0

This code below can be reassuring that the environment variable is configured properly. As Ruben states below, good information on how to set log4j to ignore the warning, is to reconfigure your log4j file to only list out the package information when it is in error. Thanks for that Ruben.

import os
for name, value in os.environ.items():
    print("{0}: {1}".format(name, value))
Jamie
  • 437
  • 4
  • 15