81

There seems to be a lot of heated discussion on the net about the changes to the reduce() function in python 3.0 and how it should be removed. I am having a little difficulty understanding why this is the case; I find it quite reasonable to use it in a variety of cases. If the contempt was simply subjective, I cannot imagine that such a large number of people would care about it.

What am I missing? What is the problem with reduce()?

casperOne
  • 73,706
  • 19
  • 184
  • 253
jeremy
  • 4,421
  • 4
  • 23
  • 27

5 Answers5

80

As Guido says in his The fate of reduce() in Python 3000 post:

So now reduce(). This is actually the one I've always hated most, because, apart from a few examples involving + or *, almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what's actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it's better to write out the accumulation loop explicitly.

There is an excellent example of a confusing reduce in the Functional Programming HOWTO article:

Quick, what's the following code doing?

total = reduce(lambda a, b: (0, a[1] + b[1]), items)[1]

You can figure it out, but it takes time to disentangle the expression to figure out what's going on. Using a short nested def statements makes things a little bit better:

def combine (a, b):
    return 0, a[1] + b[1]

total = reduce(combine, items)[1]

But it would be best of all if I had simply used a for loop:

total = 0
for a, b in items:
    total += b

Or the sum() built-in and a generator expression:

total = sum(b for a,b in items)

Many uses of reduce() are clearer when written as for loops.

Dzinx
  • 55,586
  • 10
  • 60
  • 78
  • 4
    In that case it could be even easier: sum (b for a, b in items) – John Millikin Oct 08 '08 at 07:46
  • Ah, frightening -- now that I look in the link, the sum() line is directly your quote. Must have just missed it the first time. – John Millikin Oct 08 '08 at 07:59
  • @John Millikin: You're right! I missed it, too :) I added it now, thanks! – Dzinx Oct 08 '08 at 08:22
  • 32
    The reduce code is not equivalent to the for loop. Here is the equivalent: `total = reduce(lambda total, (a,b): total + b, items, 0)` – Nathan Shively-Sanders Oct 08 '08 at 13:51
  • @Nathan Sanders: If you care only about the result, they are the same. If you care about underlying implementation, your reduce is still different from the for loop, as it uses one more variable (total). – Dzinx Oct 08 '08 at 16:15
  • 4
    "The reduce function can obfuscate code's intent. Therefore, reduce should be removed from the language." Granted, I don't know that anyone is saying exactly that, but it *is* a non sequitur. – Tony Apr 15 '14 at 17:35
  • 2
    Point taken, but to be fair, tossing in a lambda into your example is the equivalent to handing a blood-stained shirt to one guy in the police lineup. I won't argue that reduce() isn't confusing at first, but everything in functools is when you start. I'm finding functools more and more useful as I learn to use them. Now, if you want examples of confusing, check out itertools(). I'm learning to love them too, but was a rough first date. :-) – JS. Apr 26 '14 at 00:04
  • @DzinX : the for loop does use total variable inside the loop so it is fair to say that it is equivalent, but in any case, the example is confusing for those who don't know reduce because in the reduce function, it seem like you are adding `a` and `b` together while in the for loop, it seem like you are adding `b` to `total` and doesn't do anything with `a`, if it was possible to be done like what nathan-sanders said or something close to it then there would be no confusion. – freeforall tousez Feb 20 '15 at 06:35
  • 4
    The argument is "Let's move `reduce` because it can be used to write unclear code.". Well, so can `**`, `+` and `-`. Should we move them to `functools` ? – Eric Duminil Apr 23 '17 at 13:07
  • 1
    Fine. I don't appreciate anyone dictating what I find useful. So I will right my own if it is removed. –  Aug 09 '17 at 07:07
  • 1
    This isn't even a good example of a confusing lambda; the code is clearly written to look bad! It should be `sum(values)`, which could be `reduce(operator.add, values)`, and `values` could be `map(operator.itemgetter(1), items)`. Notably absent is an argument about `reduce` itself. – Yann Vernier Feb 28 '18 at 11:42
  • That reduce code looks like an intentional obfuscation. Assuming that `items` is a sequence of pairs (tuples of length 2), here is a better way to write it: `reduce(lambda a, b: a[1] + b[1], items)`. Just add the second elements. – W. Zhu Oct 02 '20 at 02:13
  • 1
    Minor note: As of 3.8, [the `math` module spawned a `math.prod` function](https://docs.python.org/3/library/math.html#math.prod), so the commonly needed use of `reduce` that `sum` didn't fulfill (to perform multiplicative reduction) is covered by the standard library; `math.prod(iterable)` is faster and simpler than either `reduce(lambda x, y: x * y, iterable, 1)` or the faster, but more imports needed, `reduce(operator.mul, iterable, 1)`. Change `iterable` to a generator expression when `reduce`'s function was also unpacking values. – ShadowRanger Dec 08 '20 at 16:09
  • 'from functools import reduce' and 'items = [('a', 1), ('b', 2)]' completes the example. – Tom J Aug 04 '22 at 02:41
39

reduce() is not being removed -- it's simply being moved into the functools module. Guido's reasoning is that except for trivial cases like summation, code written using reduce() is usually clearer when written as an accumulation loop.

John Millikin
  • 197,344
  • 39
  • 212
  • 226
  • 36
    Gah, that's horrible reasoning :( – TraumaPony Oct 08 '08 at 07:22
  • 25
    Is it? Much of Python's philosophy is about writing code that is clear and obvious. A typical call to reduce() usually requires me to break out a pencil and graph what the function is being called with. – John Millikin Oct 08 '08 at 07:57
  • 35
    Unless you can show me a *significant* performance increase (2x at least), I'll take "clear and obvious" over "compactness of expression" any day. – Kevin Little Oct 08 '08 at 14:37
9

People worry it encourages an obfuscated style of programming, doing something that can be achieved with clearer methods.

I'm not against reduce myself, I also find it a useful tool sometimes.

Eli Bendersky
  • 263,248
  • 89
  • 350
  • 412
4

The primary reason of reduce's existence is to avoid writing explicit for loops with accumulators. Even though python has some facilities to support the functional style, it is not encouraged. If you like the 'real' and not 'pythonic' functional style - use a modern Lisp (Clojure?) or Haskell instead.

Terminus
  • 925
  • 10
  • 23
  • This is like telling Chinese Americans to go back to China if they don't like how they are being treated here in the US. We are all here to make a widely used language better. No need to ask people to go use a different language. Most of the time they can't because constraints such as the availability of packages. – episodeyang May 13 '21 at 23:51
  • 1
    @episodeyang that's a strange metaphor with a lot of emotional baggage to bring into a technical discussion. Python isn't made "better" (or worse) by functional paradigms, it's simply [not a functional language by the statement of its own creator](https://stackoverflow.com/questions/1017621/why-isnt-python-very-good-for-functional-programming). There's nothing wrong with calling a spade a spade and advising folks to stop shoehorning Lisp/Clojure/Haskell idioms into a multi-paradigm, fundamentally imperative, non-functional language like Python. – ggorlen Aug 25 '21 at 22:13
  • So to me, reduce is a special case of a loop. It's a loop where the input is List[T] and the output is T. I feel the example above is unnatural because the result of the loop is different, but they use a trick to embed it in T (they use `(0, acc)` for the accumulator). My use case at the moment is to calculate `a ++ b ++ c ++ d ++ e` without either having to i. create a "zero" in T or `b` mess around with "loop and a half logic" (`acc = loop[0]; for x in loop[1:]: acc = acc ++ x`) – Att Righ Nov 04 '22 at 17:00
-2

Using reduce to compute the value of a polynomial with Horner's method is both compact and expressive.

Compute polynomial value at x. a is an array of coefficients for the polynomial

def poynomialValue(a,x):
   return reduce(lambda value, coef: value*x + coef, a)
user1153980
  • 467
  • 1
  • 4
  • 6
  • 1
    This isn't really attempting to answer the OP's question. "Why is `reduce` a problem?" Answer: "Here's a use case for `reduce`". – ShadowRanger Dec 08 '20 at 16:04
  • A powerful language tool can always be used to give brevity but lack clarity. I wanted to give an example where reduce could be used to compactly express an algorithm. Note that this is an example of an operator that is not associative. – user1153980 Dec 10 '20 at 12:21
  • Nobody is arguing `reduce` can _never_ be used clearly or even that there aren't plenty of examples where it's a great fit. The question is what's wrong with it, not what's right with it. The arguments against reduce are that it tends to be misapplied, harms readability and doesn't fit in with the [non-functional orientation of Python](https://stackoverflow.com/questions/1017621/why-isnt-python-very-good-for-functional-programming). These are the sort of possible answers to OP's question as to why it was relegated to `functools`. – ggorlen Aug 25 '21 at 17:56