0

How to set the speculative execution in hadoop? in what configuration files we need to go and make these changes? please elaborate in step wise.

Does the speculative execution changes should be made in Name node or in Data node? if in Data node can we set them to the particular systems like (1-10 systems in my cluster an i set speculative execution for 2,3 data node systems only)

Chevuri Omkar
  • 13
  • 2
  • Possible duplicate of [Hadoop speculative task execution](http://stackoverflow.com/questions/15164886/hadoop-speculative-task-execution) – Rahul Sharma Apr 09 '17 at 20:16

1 Answers1

1

Speculative Execution is turned on by default in Hadoop.

To enable/disable speculative execution, modify these properties in mapred-site.xml

<property>
   <name>mapreduce.map.speculative</name>
   <value>true</value>
</property>
<property>
   <name>mapreduce.reduce.speculative</name>
   <value>true</value>
</property>

This is a cluster-wide property and also can be set at Job level.

if in Data node can we set them to the particular systems

No, It is possible to restrict a job from running speculative tasks but not the nodes.

franklinsijo
  • 17,784
  • 4
  • 45
  • 63