Hadoop number of Reducers

Question

I am using Hadoop but when I start my job execution mappers are spawned as per number of inputs (which of course is the desired operation) but Hadoop spawns only 1 reducer regardless of input. Although there is a valid input split I don't know why Hadoop spawns only 1 reducer for the task.

Before enforcing more reducers could someone give me a hint as to why this occurs?

See this http://stackoverflow.com/questions/6885441/setting-the-number-of-map-tasks-and-reduce-tasks — Vikas Hardia, Mar 02 '14 at 14:37

score 0 · Answer 1 · answered Mar 01 '14 at 19:27

0

Check if the configuration for the job (either some XML conf file, or something in your driver) contains the property

mapred.reduce.tasks=1

Some of the example jobs have this configured by default.

answered Mar 01 '14 at 19:27

Judge Mental

5,209
17
22

score 0 · Answer 2 · answered Mar 04 '14 at 12:12

0

By default Hadoop uses only 1 reducer irrespective of the size of the input data. Here is how to up the number of reducers.

answered Mar 04 '14 at 12:12

Praveen Sripati

32,799
16
80
117

Hadoop number of Reducers

2 Answers2