51

and and or return the last element they evaluated, but why doesn't Python's built-in function any?

I mean it's pretty easy to implement oneself like this, but I'm still left wondering why.

def any(l):
    for x in l:
        if x:
            return x
    return x

edit:

To add to the answers below, here's an actual quote from that same mailing list of ye mighty emperor on the issue:

Whether to always return True and False or the first faling / passing element? I played with that too before blogging, and realized that the end case (if the sequence is empty or if all elements fail the test) can never be made to work satisfactory: picking None feels weird if the argument is an iterable of bools, and picking False feels weird if the argument is an iterable of non-bool objects.

Guido van Rossum (home page: http://www.python.org/~guido/)

Dan D.
  • 73,243
  • 15
  • 104
  • 123
doda
  • 511
  • 1
  • 5
  • 8

7 Answers7

50

This very issue came up up on the Python developer's mailing list in 2005, when Guido Van Rossum proposed adding any and all to Python 2.5.

Bill Janssen requested that they be implemented as

def any(S):
    for x in S:
        if x:
            return x
    return S[-1]

def all(S):
    for x in S:
        if not x:
            return x
    return S[-1]

Raymond Hettinger, who implemented any and all, responded specifically addressing why any and all don't act like and and or:

Over time, I've gotten feedback about these and other itertools recipes. No one has objected to the True/False return values in those recipes or in Guido's version.

Guido's version matches the normal expectation of any/all being a predicate. Also, it avoids the kind of errors/confusion that people currently experience with Python's unique implementation of "and" and "or".

Returning the last element is not evil; it's just weird, unexpected, and non-obvious. Resist the urge to get tricky with this one.

The mailing list largely concurred, leaving the implementation as you see it today.

Steven Rumbalski
  • 44,786
  • 9
  • 89
  • 119
  • 6
    Here's another nugget from the quoted discussion: Perl returns the last true/false value as well, and it is a subtle trap. I worked at a perl shop that had a long style guide which outlawed using that side effect but people did anyway. I'm +1 on having it return a true boolean for the same reason "if (x = calc_foo()):" isn't supported, if there is a quirky side effect available someone will use it and someone else won't notice. (http://mail.python.org/pipermail/python-dev/2005-March/052019.html) – Steven Rumbalski Apr 16 '12 at 20:11
  • 3
    Plus, returning the last element wouldn't work on an empty sequence. – dan04 Apr 16 '12 at 20:43
  • 1
    @dan04: Weeble's answer addresses that point nicely, so I'll leave that out of my answer for now. – Steven Rumbalski Apr 16 '12 at 20:47
  • 1
    "if there is a quirky side effect available someone will use it and someone else won't notice."--To avoid that factors out a few languages altogether--and I wholeheartedly agree. – Bill K Apr 16 '12 at 23:30
22

and and or can be sensibly defined in a way that they always return one of their operands. However, any and all cannot sensibly be defined always to return a value from their input sequence: specifically they cannot do so when the list is empty. Both any and all currently have a well defined result in this situation: any returns False and all returns True. You would be forced to sometimes return a boolean value and sometimes return an item from the sequence, which makes for an unpleasant and surprising interface. Much better to be simple and consistent.

Weeble
  • 17,058
  • 3
  • 60
  • 75
  • 2
    I think *any* answer from `any()` or `all()` is a bit surprising. – Ned Batchelder Apr 16 '12 at 19:45
  • 7
    You will set off the metal detector if `any(thing.ismetal for thing in yourpockets)`. You can post a soft bag through a slot if `all(item.maxdimension<=slot.size for item in bag)`. There are plenty of situations where the answer from `any([])` or `all([])` is entirely natural and unsurprising. – Weeble Apr 16 '12 at 19:58
  • 1
    I'd think `any` could return `None` if the list is empty. But it's probably cleaner to simply return a bool and as you said an easier interface to deal with. – doda Apr 16 '12 at 20:00
  • @doda So what would `all` return? It needs to both match `any` and be `True` in a boolean context. – agf Apr 16 '12 at 20:10
  • 1
    @agf I'd use `True`, I don't see why it has to perfectly match `all`, they're different functions after all. I don't have a problem with the design decision of python's core team, after all Python makes it simple to roll your own if you so desire. – doda Apr 16 '12 at 20:17
  • I disagree. If `sum` can have a default `start` value of `0`, why couldn't `all` have a default `start` value of `True`? – Reinstate Monica Apr 16 '12 at 21:02
17

Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), we can alternatively explicitly capture a witness of an any expression or a counterexample of an all expression:

To quote a couple examples from the PEP description:

if any(len(long_line := line) >= 100 for line in lines):
  print("Extremely long line:", long_line)

if all((nonblank := line).strip() == '' for line in lines):
  print("All lines are blank")
else:
  print("First non-blank line:", nonblank)
Xavier Guihot
  • 54,987
  • 21
  • 291
  • 190
5

I asked this same question on python-ideas, and was told the reason was that any() and all() need to return a value when the sequence is empty, and those values must be False and True. This seems like a weak argument to me.

The functions can't change now, but I think they would be more useful, and better analogs of the and and or operators they generalize, if they returned the first true-ish or false-ish value they encountered.

Willem Van Onsem
  • 443,496
  • 30
  • 428
  • 555
Ned Batchelder
  • 364,293
  • 75
  • 561
  • 662
  • I think this is backwards. Any use case for `and` or `or` returning a value can be replaced with the conditional expression. Just because they're overloaded doesn't mean other things should be. – agf Apr 16 '12 at 19:41
  • @agf The `and` and `or` behavior predates the conditional expression. – phihag Apr 16 '12 at 19:44
  • @phihag Yes, I know that. That's why they're overloaded. It makes sense for historical reasons. That doesn't mean "There should be only one way to do it" should be violated everywhere. Logically, `any` is a boolean question, so any other type of return value is unexpected. – agf Apr 16 '12 at 19:45
  • @Ned Batchelder: See my answer for historical note. – Steven Rumbalski Apr 16 '12 at 20:17
3

The behavior of and and or exists for historical reasons.

Before Python had a ternary operation / conditional expression, you used and and or if you wanted to use a value on a condition. Any such expression can be rewritten with the conditional expression syntax:

true_val if condition else false_val

Essentially, they are overloaded with two functions, and for compatibility reasons, they haven't been changed.

That is not a reason to overload other operations. any seems like it should tell you whether or not a condition is true for any item, which is a boolean, so it should return a bool.

agf
  • 171,228
  • 44
  • 289
  • 238
  • 2
    +1. This dove-tails nicely with my answer. I think that perhaps `and` and `or` not returning boolean is for more than just backwards compatability. If my reading between the lines of PEP 285 is correct, Guido seems to prefer the non-purity of `and` and `or`. – Steven Rumbalski Apr 16 '12 at 20:21
0

Any returns a boolean because it effectively treats its argument as a list of bools before considering if any of them are true. It is returning the element it evaluates, but this happens to be a bool.

When would you want to use your version of any? If it's on a list of bools then you already have the correct answer. Otherwise you are just guarding against None and might be expressed as:

filter(lambda x: x != None, l)[0]

or:

[x for x in l if x != None][0]

Which is a clearer statement of intent.

dawg
  • 98,345
  • 23
  • 131
  • 206
cmh
  • 10,612
  • 5
  • 30
  • 40
  • `any` doesn't convert to `bool` any more or less than `and` and `or`. That's not it. – agf Apr 16 '12 at 19:32
  • `and` and `or` do treat their arguments as `bool`. They are exceptional in the fact that they return one of their arguments. – cmh Apr 16 '12 at 19:36
  • 1
    But why are they exceptional and not `any`? – agf Apr 16 '12 at 19:36
  • Presumably because there would be no benefit and it allows more optimisations in the implementation. – cmh Apr 16 '12 at 19:38
  • *Speculatively*, rather than presumably. – hexparrot Apr 16 '12 at 19:39
  • It's not hard to see how dropping the requirement of returning an element allows for potentially more efficient implementation (e.g. anding a list of ints rather than comparing). I do concede that my presumption was speculative. – cmh Apr 16 '12 at 20:00
0

It's not immediately obvious that any's value could be either False or one of the values in the input. Also, most uses would look like

tmp = any(iterable)
if tmp:
   tmp.doSomething()
else:
   raise ValueError('Did not find anything')

That's Look Before You Leap and therefore unpythonic. Compare to:

next(i for i in iterable if i).doSomething()
# raises StopIteration if no value is true

The behavior of and and or was historically useful as a drop-in for the then-unavailable conditional expression.

phihag
  • 278,196
  • 72
  • 453
  • 469