How to set number of Map tasks in hadoop Job class

Question

I have a problem to set number of mappers to one through Java. I am using Job and configuration class. I know that we can set the number of reducers through mapred.map.tasks with old deprecated API.

Is there any way to specify that using Job and configuration classes. I tried by increasing the input file split size using FileInputFormat.setMinInputSplitSize(FullFilesize)

But, I would like to know do we have any equivalent to the mapred.map.tasks in the new API or how do you specify by other way

do you want all your input to go to one mapper or you want only 1 mapper to run at a time? — vishnu viswanath, Nov 17 '15 at 04:01
Which of the above 2? All input to 1 job or run only one mapper at a time — vishnu viswanath, Nov 17 '15 at 17:50
check these : http://stackoverflow.com/questions/8831303/one-mapper-or-a-reducer-to-process-one-file-or-directory/8831737#comment55266183_8831737 http://stackoverflow.com/questions/10969517/hadoop-non-splittable-textinputformat — vishnu viswanath, Nov 17 '15 at 22:51

score 0 · Answer 1 · edited May 23 '17 at 12:01

0

The number of map tasks for a given job is driven by the number of input splits and not by the mapred.map.tasks parameter

refer following stack over flow question Setting the number of map tasks and reduce tasks

edited May 23 '17 at 12:01

Community

1
1

answered Nov 16 '15 at 22:30

Balaji Thummalapenta

131
4

How to set number of Map tasks in hadoop Job class

1 Answers1