Why is using reduce with non-associative operators considered bad style?

Question

As Guido says in his The fate of reduce() in Python 3000 post:

So now reduce(). This is actually the one I've always hated most, because, apart from a few examples involving + or *, almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what's actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it's better to write out the accumulation loop explicitly.

I'm confused by the statement:

the applicability of reduce() is pretty much limited to associative operators

Isn't reduce in python just an lfold operation (left to right)? I get that order can be non-deterministic if used with sets or dict keys -- is that the only reason?

Here is an example of a nice usage of reduce for a non-associative operation:

def pass_thru(prev, next_func):
    return next_func(prev)

def flow(seed, *funcs):
    return reduce(pass_thru, funcs, seed)

flow('HELLO',
     str.lower,
     str.capitalize,
     str.swapcase)
#returns 'hELLO'

What concerns me is if I should worry about a non-guaranteed conformance to lfold, future multi-threaded implementations or just watch out for non-deterministic iterables like sets and dict.

Of course - your specific example is much clearer as: `'HELLO'.lower().capitalize().swapcase()`... — Jon Clements, Jul 30 '17 at 19:21
Sure, until I need to extend string with custom operations like compress, encrypt, sign... — Doug Coburn, Jul 30 '17 at 19:24
True... however,writing `for f in (compress, encrypt, sign): s = f(s)` isn't a big deal either... (and arguably is more explicit as to what's happening to `s`) — Jon Clements, Jul 30 '17 at 19:26
(also - you can do other related stuff inside the loop - or handle specific exceptions etc..) — Jon Clements, Jul 30 '17 at 19:29
(note - I'm not against `reduce` - heck there's `itertools.accumulate` these days, but I can kind of see the argument for outside simple cases, what it may do isn't immediately obvious (especially if mutating) or requiring default values - see: https://stackoverflow.com/questions/23011146/recursive-access-to-dictionary-and-modification - while it's obvious when you've grokked it, it's probably not as grokkable as an explicit for-loop) — Jon Clements, Jul 30 '17 at 19:30
If nothing else - using `for f in (a, b, c): s = f(s)` you can at least shove a `print` statement in there to track what's going on when things don't seem right :) — Jon Clements, Jul 30 '17 at 19:37
@JonClements I'm not trying to argue that reduce should stay or is better than a for loop. [This](https://stackoverflow.com/questions/181543) question discusses that point. I'm just trying to understand the associativity argument. — Doug Coburn, Jul 30 '17 at 19:55
@StefanPochmann Thanks for the suggestion -- I updated the question text. Hopefully it is more clear now. — Doug Coburn, Jul 31 '17 at 17:19
@JonClements I've been thinking about your alternative : `for f in (a, b, c): s = f(s)` . I think this _is_ the pythonic way. The left-to-right iterative nature of the implementation is concise and clear. Which, I think, leads to the associativity argument -- unless the operation is associative, you have to know the implementation details or reduce to understand if the code is correct. — Doug Coburn, Jul 31 '17 at 17:29
@DougCoburn nitpick: it's not really an implementation detail, it is part of the definition of a lfold, which is what reduce is. This could be implemented in a variety of ways. I think Guido's point is that *unless* you have an associative operation, then it isn't easy to reason about unless you are very familiar with the functional paradigm, but Python, while allowing for functional constructs, is a fundamentally imperative language. Indeed, I think Python is great because it makes imperative code so clean. Part of that is it has functional constructs which you can use when it makes sense. — juanpa.arrivillaga, Aug 03 '17 at 17:52

score 1 · Accepted Answer · answered Jul 30 '17 at 19:42

While I have to say I agree with you, and I really liked the flow example you've presented,

I can see some merit to Guido's claims that reduce is not as explicit as a direct loop. And it might be confusing if you are not used to FP's higher order functions.

I think that higher order function such as map , filter and zip are very useful, they are declarative by nature, and do not define the explicit implementation.

You can use spark's map, the multiprocessing.Pool's map or the built-in map, and the return value should be the same.

However, when it comes to reduce, the implementation is serial by nature. and in fact, you do not gain any abstraction over using a direct loop.

Thus, I can see how reduce may conflict with the Zen of Python:

There should be one-- and preferably only one --obvious way to do it.

Why is using reduce with non-associative operators considered bad style?

1 Answers1