Need help for installing hadoop

Question

I am new to Hadoop and trying to install it on my windows 7 (x64) PC using cygwin. I followed this tutorial to install http://alans.se/blog/2010/hadoop-hbase-cygwin-windows-7-x64/#software

Everything goes correct before this step -

Only Hadoop 0.21.0: Next, one line has to be added to the hadoop-config.sh file in hadoop-0.21.-0/bin

    CLASSPATH=`cygpath -wp "$CLASSPATH"`

Add this line before the line containing

    JAVA_LIBRARY_PATH=''

Does this line CLASSPATH=cygpath -wp "$CLASSPATH" need to be copied as it is?

Coz when I am trying to run command below.

    $ bin/hadoop namenode -format

I am getting such error from cygwin

/usr/local/hadoop-0.21.0/bin/hadoop-config.sh: line 15: $'\r': command not found
/usr/local/hadoop-0.21.0/bin/hadoop-config.sh: line 19: $'\r': command not found
/usr/local/hadoop-0.21.0/bin/hadoop-config.sh: line 21: $'\r': command not found
/usr/local/hadoop-0.21.0/bin/hadoop-config.sh: line 75: syntax error near unexpected token `$'in\r''
'usr/local/hadoop-0.21.0/bin/hadoop-config.sh: line 75: `case "`uname`" in 
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

HDFS not found.

can anyone explain what is going wrong?

Did you try using the recommended command? What's your `$CLASSPATH`? — Dave Newton, Feb 06 '12 at 18:44
@DaveNewton: The tutorial says something like this... The reason for this is that in order for the CLASSPATH to be build with all the Hadoop jars (line ~120 to ~200) the path needs to be in the Cygwin format (/cygdrive/c/cygwin/usr/local/hadoop…), however in order for Java use the classpath, it needs to be in the Windows format (c:\cygwin\usr\local\hadoop..). The line transforms the Cygwin built classpath to one that is understood by Windows. what does this mean?? — Nilesh Barai, Feb 07 '12 at 06:08

user1978601 · Answer 1 · 2013-08-04T17:04:02.523

0

Convert the config.sh file or any config file needed in d build to unix format with dos2unix filename (shell command). It seems the files changes to DOS format by default. Better convert every file in d folder to unix with (dos2unix ./*) shell command.

edited Aug 04 '13 at 17:04

answered Aug 03 '13 at 05:53

user1978601

53
1
7

score 0 · Answer 2 · edited May 23 '17 at 12:11

0

The error with Cygwin seems to be a known error with npm script. You can use the work around mentioned here,

https://stackoverflow.com/a/22436199/2390020

edited May 23 '17 at 12:11

Community

1
1

answered Mar 21 '14 at 00:31

Tharaka

2,355
1
20
22

score 0 · Answer 3 · answered Feb 06 '12 at 20:49

0

you are using a deprecated command. For a list of hadoop commands, enter hadoop on your console. I believe the command is hadoop namenode -format

answered Feb 06 '12 at 20:49

RFT

1,041
4
13
25

@DaveNewton: thats where i am confused... in this line CLASSPATH=cygpath -wp "$CLASSPATH" am I supposed to write the path of the hadoop folder in windows by replacing $CLASSPATH and path of hadoop in cygwin by replacing cygpath ?? – Nilesh Barai Feb 07 '12 at 05:54

score 0 · Answer 4 · answered Mar 18 '12 at 01:40

I also got these errors using cygwin 1.7:

/usr/local/hadoop-0.21.0/bin/hadoop-config.sh: line 15: $'\r': command not found

It appears that these files are not in DOS CR/LF format instead of the expected UNIX format.

You can confirm this with a text editor like Notepad++. In the lower right hand part of the window, it can say Dos\Windows (very likely what you would see) or UNIX.

To work around these errors, in a cygwin window I used the command dos2unix *.sh in the bin folder and other folders (conf, etc...) as needed.

Hope this helps you!

Need help for installing hadoop

4 Answers4