0

I have basically this problem, except that when I go to the folder where hadoop is installed, there is no hadoop/config, so I don't know where to look for hadoop-env.sh. How can I get Hadoop to look for my up-to-data Java rather than my mandatory OSX legacy Java?

Community
  • 1
  • 1
eric_kernfeld
  • 495
  • 5
  • 17

2 Answers2

1

I've tried hadoop-2.7.2 locally, in my experience, you can specify your configuration directory by environment HADOOP_CONF_DIR.

To put it simple, you can just export HADOOP_CONF_DIR before you start HDFS by start-dfs.sh. At this point, you can add your own JAVA_HOME at $HADOOP_CONF_DIR/hadoop-env.sh. By the way, $HADOOP_CONF_DIR usually points to a directory copy from the hadoop release. For example, in hadoop-2.7.2 you can just copy hadoop-2.7.2/etc/hadoop/ and update the necessary configuration and specify it as your own HADOOP_CONF_DIR.

Thanks for eric_kernfeld's improvement:

UPDATE: since OP used a Homebrew installation, there were multiple directories named hadoop, and OP had to go through <version>/libexec/etc/hadoop.

At the end, hadoop-2.7.2/etc/hadoop/(please use your own version instead of 2.7.2) is the default HADOOP_CONF_DIR, which means you can just update hadoop-2.7.2/etc/hadoop/hadoop-env.sh as you wish.

However, there might be some difference if your hadoop release early than 2.x.x.

Community
  • 1
  • 1
Horsing
  • 1,070
  • 7
  • 22
  • Since there are multiple directories named `hadoop`, and the `etc` helped me find the right one, I am accepting this answer. But, I had to go through `/libexec/etc/hadoop`, not `/etc/hadoop` as you indicate. May I edit your answer to include this? – eric_kernfeld Apr 24 '16 at 18:18
  • I'm so appreciate that you could improve it. Thank you very much! – Horsing Apr 25 '16 at 01:21
  • Actually, I've ignored the fact that you ware not using the tarball of hadoop release but from the operating system source repository. In my tries, I've only downloaded the tarball from official [site](http://apache.fayea.com/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz) and extract it and update the necessary environment. In my circumstance, it's actually in directory `hadoop-x.x.x/etc/hadoop`. I'm sorry for not specifying it clearly. – Horsing Apr 25 '16 at 01:27
0

For Hadoop 2+, the hadoop-env.sh file should be directly in hadoop/hadoop-env.sh.

If the file not exist, you could create it, and put inside:

export JAVA_HOME={your java home path}
JosEduSol
  • 5,268
  • 3
  • 23
  • 31