1

Running a hadoop job, I set my map to 100 and my reduce to 1 (I am not really reducing anything).

conf.setNumMapTasks(100);
conf.setNumReduceTasks(1);

After the job runs, I look at the hadoop log I see that one reduce task was used and that zero (0) map task was used. Now I have two questions:

1) Even if my job is very small, shouldn't the number of map task be at least one?

2) Is there a way to force hadoop to run a specified number of map task? I already know that conf.setNumMapTasks(100) is just a hint. But is there another way?

user1467855
  • 3,843
  • 7
  • 28
  • 29
  • You might want to take a look at this [thread][1]. [1]: http://stackoverflow.com/questions/6885441/setting-the-number-of-map-tasks-and-reduce-tasks – Ambar Sep 09 '12 at 08:38
  • Actually I read it before posting – user1467855 Sep 09 '12 at 19:43
  • @Ambar: I am particularly interested in the answer to question 1: shouldn't I end up with at least one map (and of course evidence/reference for that). Presently I am getting zero(0) map tasks. – user1467855 Sep 09 '12 at 19:48

0 Answers0