8

Working with somebody else's code I stumbled across this gotcha. So what is the explanation for numpy's behavior?

In [1]: import numpy as np

In [2]: foo = [False, False]

In [3]: print np.any(x == True for x in foo)
True  # <- bad numpy!

In [4]: print np.all(x == True for x in foo)
True  # <- bad numpy!

In [5]: print np.all(foo)
False  # <- correct result

p.s. I got the list comprehension code from here: Check if list contains only item x

Community
  • 1
  • 1
Framester
  • 33,341
  • 51
  • 130
  • 192
  • 5
    These are not technically list comprehensions, rather generator expressions. – Lev Levitsky May 02 '13 at 09:53
  • 1
    possible duplicate of [numpy all differing from builtin all](http://stackoverflow.com/questions/14391501/numpy-all-differing-from-builtin-all) – wim May 02 '13 at 10:07
  • 6
    This is a real "gotcha" for people using ipython's "pylab mode", which overwrites the built-in any and all with the numpy any and all. So you can have a code snippet that works in pure python but fails in pylab mode. – Steve Byrnes May 02 '13 at 13:31
  • @SteveB That is exactly what happened to me. – Framester May 02 '13 at 13:54

1 Answers1

15

np.any and np.all don't work on generators. They need sequences. When given a non-sequence, they treat this as any other object and call bool on it (or do something equivalent), which will return True:

>>> false = [False]
>>> np.array(x for x in false)
array(<generator object <genexpr> at 0x31193c0>, dtype=object)
>>> bool(x for x in false)
True

List comprehensions work, though:

>>> np.all([x for x in false])
False
>>> np.any([x for x in false])
False

I advise using Python's built-in any and all when generators are expected, since they are typically faster than using NumPy and list comprehensions (because of a double conversion, first to list, then to array).

Fred Foo
  • 355,277
  • 75
  • 744
  • 836
  • 1
    the documentation is quite vague.. `Input array or object that can be converted to an array.` but still, the way I interpret it, it should work with generators... – Karoly Horvath May 02 '13 at 10:03
  • 4
    @KarolyHorvath: the `array` docstrings spells it out: "An array, any object exposing the array interface, an object whose `__array__` method returns an array, or any (nested) sequence." This is the definition of "array-like" used throughout NumPy. Unfortunately, any *other* object is converted to 0-d array (try `np.array("foo").shape`). – Fred Foo May 02 '13 at 10:07