Hadoop - Restart datanode and tasktracker

Question

I want to bring down a single datanode and tasktracker, so that some new changes that i've made in my mapred-site.xml take effect, such as mapred.reduce.child.java.opts etc. How do I do that? However I don't want to bring down the whole cluster since i have active jobs running.

Also, how can that be done ensuring that the namenode does not copy the relevant data blocks of a "temporarily down" datanode onto another node

user2486495 · Accepted Answer · 2013-11-26T05:42:55.830

15

To stop

You can stop the DataNodes and TaskTrackers from NameNode's hadoop bin directory.

./hadoop-daemon.sh stop tasktracker 
./hadoop-daemon.sh stop datanode

So this script checks for slaves file in conf directory of hadoop to stop the DataNodes and same with the TaskTracker.

To start

Again this script checks for slaves file in conf directory of hadoop to start the DataNodes and TaskTrackers.

./hadoop-daemon.sh start tasktracker
./hadoop-daemon.sh start datanode

edited Nov 26 '13 at 05:42

answered Nov 26 '13 at 05:37

user2486495

1,709
11
19

1

i wanted to stop just 1 tasktracker/datanode combination. not the whole cluster since i had jobs running. i'll clarify my question accordingly – Abbas Gadhia Nov 26 '13 at 05:43
1

In that case you need to ssh into the DataNode and execute your commands. – user2486495 Nov 26 '13 at 05:45
I found `hadoop-daemon.sh` in `$HADOOP_HOME/sbin` – Sida Zhou Jul 23 '20 at 06:29

score 10 · Answer 2 · answered Sep 02 '16 at 02:23

10

In Hadoop 2.7.2, tasktracker is long gone, to manually restart services out on slaves:

yarn-daemon.sh stop nodemanager
hadoop-daemon.sh stop datanode
hadoop-daemon.sh start datanode
yarn-daemon.sh start nodemanager

answered Sep 02 '16 at 02:23

Joel Bondurant

841
1
10
17

Glad you mentioned 2.7.2 - perhaps Hadoop questions should be tagged with a version? – Jeremy Hajek Nov 28 '16 at 02:49

score 4 · Answer 3 · answered Nov 26 '13 at 04:45

4

Ssh into the datanode/tasktracker machine and cd into the bin directory of hadoop.

Invoke

./hadoop-daemon.sh stop tasktracker

./hadoop-daemon.sh stop datanode

./hadoop-daemon.sh start datanode

./hadoop-daemon.sh start tasktracker

I'm not sure if restarting the tasktracker is required for the changes in mapred-site.xml to take effect. Please leave a comment so that i can correct my answer if needed

answered Nov 26 '13 at 04:45

Abbas Gadhia

14,532
10
61
73

This answer helped me - some of my data nodes were dead (not responding to dfsadmin -report) and SSHing into the offending node and doing this helped bring them back online. – drjrm3 Feb 04 '15 at 14:09

Hadoop - Restart datanode and tasktracker

3 Answers3

Linked