0

Entire list of parameter( for Hadoop-2.6) is given on the link

But you can execute a job in either MRv1 or MRv2 style. I think there are some parameters that are only applicable for MRv1 like mapreduce.tasktracker.map.tasks.maximum, is this true ? If yes, then is there any smarter way to figure out all such parameters ? can we pass all parameters via -Dproperty=value or -D property=value or is there any parameter, that i can not pass like this ?

Novice
  • 155
  • 3
  • 14

1 Answers1

1

In MR2, the number of map tasks will be calculated dynamically by resource manager depending on the memory assigned to the each container.

From hortonworks

With YARN and MapReduce 2, there are no longer pre-configured static slots for Map and Reduce tasks. The entire cluster is available for dynamic resource allocation of Maps and Reduces as needed by the job

This page has got more info. check this http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/

You can pass any parameter using -D property = value from command line. There is no restriction on that. But if these properties are set in code by developer, the command line property will be overidden by job.

alekya reddy
  • 934
  • 1
  • 8
  • 14
  • is there any smarter way to figure out all such parameters ? – Novice May 06 '15 at 18:38
  • 1
    you can check mr1 and correspondin mr2 properties here . http://www.cloudera.com/content/cloudera/en/documentation/core/v5-2-x/topics/cdh_ig_mapreduce_to_yarn_migrate.html – alekya reddy May 06 '15 at 18:45