Yarn NodeManager and ResourceManager in the same node

Question

(By default) Is there a "node manager" in the same node with "resource manager" in Hadoop Yarn? If not, is it possible to run them on the same node?

score 3 · Answer 1 · answered Aug 08 '13 at 21:44

Depends on whether you want to run other containers (for ApplicationMasters or tasks) on the RM node. If you want your RM node to be dedicated to only resource-management, scheduling, etc., then do not start NM on the RM node. If you think you'll have spare cpu/mem/disk on the RM node (especially for a small cluster, or a cluster with few jobs to schedule), then run a NM on the RM node.

score 2 · Accepted Answer · edited Jun 03 '14 at 12:05

2

The script sbin/start-yarn.sh does just that

# start resourceManager

"$bin"/yarn-daemon.sh --config $YARN_CONF_DIR start resourcemanager

# start nodeManager

"$bin"/yarn-daemons.sh --config $YARN_CONF_DIR start nodemanager

in the yarn-site.xml you can set the IP:PORT of the resourcemanager as well as the node manager

yarn.resourcemanager.address

yarn.nodemanager.address

The node manager is the one that locate the resource manager (see http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html)

edited Jun 03 '14 at 12:05

andyp

6,229
3
38
55

answered Jul 29 '13 at 05:48

Amir Pauker

56
4

From the link that you provided it looks like RM and NM are not in the same node. Is it common to NOT put RM and NM into the same node? – polerto Jul 30 '13 at 20:16

Yarn NodeManager and ResourceManager in the same node

2 Answers2