13

Logical operators in Python are lazy. With the following definition:

def func(s):
    print(s)
    return True

calling the or operator

>>> func('s') or func('t')
's'

only evaluates the first function call, because or recognizes that the expression evaluates to True, irregardless of the return value of the second function call. and does behave analogously.

However, when using any() (analogously: all()) in the following way:

>>> any([func('s'), func('t')])
's'
't'

all function calls are evaluated, because the inner list is constructed first, before any starts to iterate over the boolean values of its items. The same happens when we omit the list construction and just write

>>> any(func('s'), func('t'))
's'
't'

That way we lose the power of any being short-circuit, which means that it breaks as soon as the first element of the iterable is truish. If the function calls are expensive, evaluating all the functions up front is a big loss and is a waste of this ability of any. In some sense, one could call this a Python gotcha, because it might be unexpected for users trying to leverage this feature of any, and because any is often thought as being just another syntactic way of chaining a sequence of or statements. But any is just short-circuit, not lazy, and that is a difference here.

any is accepting an iterable. So, there should be a way of creating an iterator which does not evaluate its elements up front but pass them unevaluated to any and lets them evaluate inside of any only, in order to achieve a fully lazy evaluation.

So, the question is: How can we use any with truly lazy function evaluation? That means: How can we make an iterator of function calls which any can consume, without evaluating all the function calls in advance?

Jonathan Scholbach
  • 4,925
  • 3
  • 23
  • 44
  • 2
    There are all manner of iterator producing things in Python. For example `map()`: `any(map(func, ('s', 't')))`. Most people seem to have the opposite problem in Python3 — they want list outputs and python gives them generators and maps! – Mark Sep 27 '20 at 16:40
  • 2
    This has nothing to do with `any`. The evaluations happen while constructing the list. – Niklas Mertsch Sep 27 '20 at 17:57

2 Answers2

16

We can use a generator expression, passing the functions and their arguments separately and evaluating only in the generator like so:

>>> any(func(arg) for arg in ('s', 't'))
's'

For different functions with different signatures, this could look like the following:

any(
    f(*args)
    for f, args in [(func1, ('s',)), (func2, (1, 't'))]
)

That way, any will stop calling the next() element in the generator as soon as one function call in the generator evaluates to True, and that means that the function evaluation is fully lazy.

Another neat way to postpone the function evaluation was mentioned by wjandrea in a comment: We can also to use lambda expressions, like so:

>>> any(
>>>    f() 
>>>    for f in [
>>>        lambda: func('s'), 
>>>        lambda: func('t'),
>>>    ]
>>>)
's'
Jonathan Scholbach
  • 4,925
  • 3
  • 23
  • 44
  • 4
    Even more broadly, you can pass expressions via lambdas: `any(e() for e in [lambda: ''.isalnum(), lambda: True, lambda: 0/0])`. This doesn't raise a `ZeroDivisionError` cause the `0/0` expression is never evaluated. – wjandrea Sep 27 '20 at 18:42
0

It is sad that any() and all() don't have the logical functionality and are limited by this somewhat artificial constraint. Instead, a once-thru loop construction can be handy, which is related also to a function with early returns.

The once-thru loop can be handy way to handle it, particularly if there are intermediate results that need to be generated and then used.

    for _ in range(1):
        val = func1()
        if not val:
            break
        val = intermediate_func1()
        if not func3(val):
            break
        val2 = intermediate_func2(val)
        if not func4(val2):
            break
        result = the_really_expensive_function(val, val2)
        if result:
            return True, result
    return False, None

similar construct using a function and early returns.

def example():

    val = func1()
    if not val:
        return False, None
    val1 = intermediate_func1(val)
    if not func3(val1):
        return False, None
    val2 = intermediate_func2(val1)
    if not func4(val2):
        return False, None
    result = the_really_expensive_function(val, val2)
    if result:
        return True, result
    return False, None

What I wanted to use (and can't; and this is only feasible with := operator):

if all(
    val := func1(),
    func3(val1 := intermediate_func1(val)),
    func4(val2 := intermediate_func2(val)),
    result := the_really_expensive_function(val, val2),
    ):
    return result, True
return False, None

Maybe in the future this will be feasible.

--Ray

Ray Lutz
  • 91
  • 5