Incorrect configuration: namenode address dfs.namenode.rpc-address is not configured

Question

I am getting this error when I try and boot up a DataNode. From what I have read, the RPC paramters are only used for a HA configuration, which I am not setting up (I think).

2014-05-18 18:05:00,589 INFO  [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(572)) - DataNode metrics system shutdown complete.
2014-05-18 18:05:00,589 INFO  [main] datanode.DataNode (DataNode.java:shutdown(1313)) -     Shutdown complete.
2014-05-18 18:05:00,614 FATAL [main] datanode.DataNode (DataNode.java:secureMain(1989)) - Exception in secureMain
java.io.IOException: Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
at org.apache.hadoop.hdfs.DFSUtil.getNNServiceRpcAddresses(DFSUtil.java:840)
at   org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.refreshNamenodes(BlockPoolManager.java:151)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:745)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:278)

My files look like:

[root@datanode1 conf.cluster]# cat core-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>

<property>
 <name>fs.defaultFS</name>
 <value>hdfs://namenode:8020</value>
</property>

</configuration>

cat hdfs-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
<property>
 <name>dfs.datanode.data.dir</name>
 <value>/hdfs/data</value>
</property>
<property>
 <name>dfs.permissions.superusergroup</name>
 <value>hadoop</value>
</property>
</configuration>

I am using the latest CDH5 distro.

Installed Packages
Name        : hadoop-hdfs-datanode
Arch        : x86_64
Version     : 2.3.0+cdh5.0.1+567
Release     : 1.cdh5.0.1.p0.46.el6

Any helpful advice on how to get past this?

EDIT: Just use Cloudera manager.

Is [this other question](http://stackoverflow.com/questions/14531590/dfs-namenode-servicerpc-address-or-dfs-namenode-rpc-address-is-not-configured) useful? — nelsonda, May 19 '14 at 21:30

score 24 · Answer 1 · answered Dec 08 '15 at 15:46

24

I too was facing the same issue and finally found that there was a space in fs.default.name value. truncating the space fixed the issue. The above core-site.xml doesn't seem to have space so the issue may be different from what i had. my 2 cents

answered Dec 08 '15 at 15:46

QADeveloper

469
4
6

Not sure why this was downvoted. Removing spaces between properties did solve my problem. I was following the digital ocean guide. – Divyanshu Das Mar 29 '16 at 16:27
3

Had the same issue, (caused by copying the code from [the tutorial](http://www.tutorialspoint.com/hadoop/hadoop_enviornment_setup.htm) without cleaning up) – Keerthi Bandara Jul 02 '16 at 14:27
I had a similar issue. Rather than a space, I simply had invalid information in the same place for fs.default.name. Everything worked fine after I fixed it. – Michael Galaxy Apr 24 '19 at 03:33

score 7 · Answer 2 · edited Mar 02 '16 at 21:00

7

These steps solved the problem for me:

export HADOOP_CONF_DIR = $HADOOP_HOME/etc/hadoop
echo $HADOOP_CONF_DIR
hdfs namenode -format
hdfs getconf -namenodes
./start-dfs.sh

edited Mar 02 '16 at 21:00

Mi-Creativity

9,554
10
38
47

answered Mar 02 '16 at 20:36

Hamdi Charef

639
2
12
19

score 1 · Answer 3 · answered Apr 21 '15 at 00:07

1

check the core-site.xml under $HADOOP_INSTALL/etc/hadoop dir. Verify that the property fs.default.name is configured correctly

answered Apr 21 '15 at 00:07

OPMendeavor

430
6
14

score 1 · Answer 4 · answered Jan 25 '17 at 03:39

1

Obviously,your core-site.xml has configure error.

<property>
 <name>fs.defaultFS</name>
 <value>hdfs://namenode:8020</value>
</property>

Your <name>fs.defaultFS</name> setting as <value>hdfs://namenode:8020</value>,but your machine hostname is datanode1.So you just need change namenode to datanode1 will be OK.

answered Jan 25 '17 at 03:39

cutd

229
3
11

this answer and Hamdi Charef's answer together were my solution. – Kevin Milner May 03 '19 at 15:52

score 0 · Answer 5 · edited Dec 22 '15 at 17:12

0

I had the exact same issue. I found a resolution by checking the environment on the Data Node:

$ sudo update-alternatives --install /etc/hadoop/conf hadoop-conf /etc/hadoop/conf.my_cluster 50
$ sudo update-alternatives --set hadoop-conf /etc/hadoop/conf.my_cluster

Make sure that the alternatives are set correctly on the Data Nodes.

edited Dec 22 '15 at 17:12

Manjunath Ballur

6,287
3
37
48

answered Jun 04 '14 at 01:08

user3705189

1

score 0 · Answer 6 · edited Apr 25 '18 at 18:27

0

in my case, I have wrongly set HADOOP_CONF_DIR to an other Hadoop installation.

Add to hadoop-env.sh:

export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop/

edited Apr 25 '18 at 18:27

ParisaN

1,816
2
23
55

answered Apr 10 '17 at 18:43

Yousef Irman

109
4

score 0 · Answer 7 · answered Jul 29 '17 at 07:09

0

Configuring the full host name in core-site.xml, masters and slaves solved the issue for me.

Old: node1 (failed)

New: node1.krish.com (Succeed)

answered Jul 29 '17 at 07:09

Devas

1,544
4
23
28

score 0 · Answer 8 · answered Aug 30 '17 at 13:41

0

creating dfs.name.dir and dfs.data.dir directories and configuring full hostname in core-site.xml, masters & slaves is solved my issue

answered Aug 30 '17 at 13:41

Hemantha Kumara M S

51
1
3

score 0 · Answer 9 · answered Dec 28 '17 at 09:38

0

In my situation, I fixed by change /etc/hosts config to lower case.

answered Dec 28 '17 at 09:38

LoranceChen

2,453
2
22
48

score 0 · Answer 10 · edited Jun 13 '18 at 16:32

0

This type of problem mainly arises is there is a space in the value or name of the property in any one of the following files- core-site.xml, hdfs-site.xml, mapred-site.xml, yarn-site.xml

just make sure you did not put any spaces or (changed the line) in between the opening and closing name and value tags.

Code:

 <property> 
<name>dfs.name.dir</name> <value>file:///home/hadoop/hadoop_tmp/hdfs/namenode</value> 
<final>true</final> 
</property>

edited Jun 13 '18 at 16:32

DeshDeep Singh

1,817
2
23
43

answered Jun 13 '18 at 15:24

ranubwj

1
1

1

Give some example please – DeshDeep Singh Jun 13 '18 at 15:32
you should add in your answer so that all can get it. i have done it for now. – DeshDeep Singh Jun 13 '18 at 16:26

score 0 · Answer 11 · edited Jun 24 '18 at 14:55

0

I was facing the same issue, formatting HDFS solved my issue. Don't format HDFS if you have important meta data.
Command for formatting HDFS: hdfs namenode -format

When namenode was not working

After formatting HDFS

edited Jun 24 '18 at 14:55

James Wong

4,529
4
48
65

answered Jun 24 '18 at 14:03

rahul mahajan

1
1

score 0 · Answer 12 · answered Dec 17 '18 at 11:37

0

Check your '/etc/hosts' file:
There must be a line like below: (if not, so add that)
namenode 127.0.0.1
Replace 127.0.01 with your namenode IP.

answered Dec 17 '18 at 11:37

Mohammadhosein Qanbari

1
1

score 0 · Answer 13 · answered Oct 11 '20 at 09:50

0

Add the below line in hadoop-env.cmd

set HADOOP_HOME_WARN_SUPPRESS=1

answered Oct 11 '20 at 09:50

Siddharth Sheladiya

1

score 0 · Answer 14 · answered Sep 01 '23 at 04:14

I am aware, lately answering this.

Should check following to fix :

Master is able to ping slave by name and vice-versa; If you have used host names in configuration instead of ip address. if not able to ping should review /etc/hosts on both master and slave.. add entries of all nodes in all nodes.
After all configuration changes are done on master, execute following on master. * change <USER_> and <SLAVE_NODE> to actual values.

scp $HADOOP_HOME/etc/hadoop/* <USER_>@<SLAVE_NODE>:$HADOOP_HOME/etc/hadoop/

Incorrect configuration: namenode address dfs.namenode.rpc-address is not configured

cat hdfs-site.xml

14 Answers14

Linked