dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured

Question

I was trying to configure hadoop with one name node and four data nodes. I was able to successfully configure the name node and job tracker on one machine and bring it up.

But on my machine where the data node is to be configured I did the following:

• I unzipped the hadoop-2.0.0-cdh4.1.2.tar.gz and mr1-2.0.0-mr1-cdh4.1.2.tar.gz into a folder and set up the configuration with master and slave.
• In the master file I had set up the ip address of machine that was configured to be the name node.
• In the slave machine I set up the ip address of my machine where I had unzipped the tar balls.

When I try to start the data node with the command

hdfs datanode

I get this error:

java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.

What do I need to do?

Answer 1

The name of the masters file is misleading. It should contain the address of the SecondaryNameNode and is read by the NameNode itself. DataNodes do not have anything to do with the masters file. You need to configure fs.default.name on core-site.xml configuration file.

The error you see is also misleading and points you to the wrong configuration parameter.

Answer 2

These steps solved the problem for me :

1. export HADOOP_CONF_DIR="$HADOOP_HOME/etc/hadoop"
2. echo $HADOOP_CONF_DIR
3. hdfs namenode -format
4. hdfs getconf -namenodes
5. start-dfs.sh

Then, Hadoop can properly started.

Answer 3

Adding the rpc-address in hdfs-site.xml for the name node will work like this

<property>
<name>dfs.namenode.rpc-address</name>
<value>dnsname:port</value>
</property>

also in core-site add the property

<property> <name>fs.defaultFS</name> <value>dnsname:port</value> </property>