0

I was following https://amodernstory.com/2014/09/23/installing-hadoop-on-mac-osx-yosemite/ to install hadoop on my mac for local small tasks.

When hstart/hstop, getting "WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable"

Have tried options in below link to rename folder/path for lib/native. Renamed libexec/ to lib/ but errors in scripts that libexec should be correct. Hadoop "Unable to load native-hadoop library for your platform" warning

However, in hadoop 3.0.0, the structure is like:

username-local:hadoop me$ pwd
/usr/local/Cellar/hadoop

username-local:hadoop me$ ls
3.0.0   hdfs

username-local:hadoop me$ cd 3.0.0/
username-local:3.0.0 me$ ls
INSTALL_RECEIPT.json    NOTICE.txt      bin         sbin
LICENSE.txt     README.txt      libexec

username-local:3.0.0 me$ cd libexec/
username-local:libexec me$ ls
bin etc libexec logs    sbin    share

username-local:libexec me$ cd libexec/
username-local:libexec me$ pwd
/usr/local/Cellar/hadoop/3.0.0/libexec/libexec
username-local:libexec me$ ls
hadoop-config.sh        hdfs-config.sh          tools
hadoop-functions.sh     mapred-config.sh        yarn-config.sh
hadoop-layout.sh.example    shellprofile.d

username-local:libexec me$ cd ../etc/
username-local:etc me$ ls
hadoop
username-local:etc me$ cd hadoop/
username-local:hadoop me$ ls
capacity-scheduler.xml          httpfs-log4j.properties         mapred-site.xml
configuration.xsl           httpfs-signature.secret         shellprofile.d
container-executor.cfg          httpfs-site.xml             ssl-client.xml.example
core-site.xml               kms-acls.xml                ssl-server.xml.example
hadoop-env.sh               kms-env.sh              user_ec_policies.xml.template
hadoop-metrics2.properties      kms-log4j.properties            workers
hadoop-policy.xml           kms-site.xml                yarn-env.sh
hadoop-user-functions.sh.example    log4j.properties            yarn-site.xml
hdfs-site.xml               mapred-env.sh
httpfs-env.sh               mapred-queues.xml.template
username-local:hadoop me$

Any ideas please? Thank you.

JoshDM
  • 4,939
  • 7
  • 43
  • 72
sue
  • 43
  • 4
  • First of all, that warning is safe to ignore. Secondly, don't rename or delete anything... Why don't you just export the correct file path for the native libraries (assuming Homebrew downloaded them)? – OneCricketeer Jun 04 '18 at 12:12
  • Hi @cricket_007 Thank you for the suggestions. I reverted back all the settings. I wasn't able to find the lib/native path or any folders containing 'native' to export. There's no lib/ folder in hadoop 3.0.0 mac version. When I do `jps`, the datanode is not running, so I want to fix this warning in case it's one of the bugging reasons. Could you give some suggestions on locating the correct file path for the native libraries for hadoop 3.0.0 please? – sue Jun 04 '18 at 17:55

0 Answers0