"The truth value of an array with more than one element is ambiguous" when comparing lists (not arrays)

Question

I have a question regarding the equality operator (==) regarding lists. When I run my script, I get an error in an if statement

"ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()".

The if statement looks like this

if variable_a == variable_b:

I know that that this is caused by an array of boolean values being evaluated by the if statement, however to debug I printed the inputs to the function. I know they are from the type list.

print(variable_a)
print(variable_b)

I get an output that looks like this

[[1,2],3]
[3,4]

But then I try the following in the Console

[[1,2],3] == [3,4]

which returns False. This is not an array of boolean values so I don't know why the if statement fails. I am using python 3.9.

Edit: I just ran

print(type(variable_a))
print(type(variable_b))

which both returned

<class 'list'>

So I am definitely sure that I am comparing lists and not numpy arrays or pandas dataframes.

Answer 1

This is a curious interaction between the built-in list type's comparison, and Numpy's comparison - which is implemented by each Numpy datatype, not just arrays.

A truly minimal example:

>>> import numpy as np
>>> a = np.int64(1)
>>> [a] == [[a, a]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The comparison involves the following chain of events:

• [a] and [[a, a]] are both lists, so the list __eq__ method is used.
• First, the identities and lengths of the lists are checked. They're separate lists of the same length, so the code can't bail out early.
• So we must compare the lists elementwise, and check that each pair is equal.
• The first elements of the lists are a and [a, a], so we compare those.
• The call a.__eq__([a, a]) succeeds: a is a Numpy scalar, and [a, a] is iterable, so Numpy broadcasts the comparison across the list.
• Now the overall list comparison needs to check whether the equality comparison returned a truthy result.However, the result of the __eq__ call was a Numpy array (array([True, True])), which doesn't allow conversion to boolean (i.e., the standard problem). So an exception is raised.

But notice:

>>> [a] == [[a]]
True

When a is compared to [a], the resulting array has exactly one total element. Numpy treats this case specially.

>>> [[a]] == [[a, a]]
False

When [a] is compared to [a, a], both are lists, so Numpy doesn't get to interfere yet. The list equality method is called again, and it checks the lengths of the lists first. Since the lengths differ, False is returned before any Numpy broadcasting can occur.

Also of note:

>>> [[a, a]] == [a]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Even though the list is clearly "not equal to" the Numpy scalar, the list comparison operation will return NotImplemented rather than False, allowing the Numpy scalar's __eq__ to be tried after that.

>>> [1] == [[a,a]]
False

This succeeds because the ordinary integer 1 will be compared to a list of Numpy objects, and the integer doesn't implement any broadcasting.

>>> [a] == [[1, 1]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The Numpy scalar will be compared to a list of integers, and broadcast across that; and the scalar value is indeed comparable to integer.

>>> [a] == [['a', 'b']]
<stdin>:1: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
False
>>> [a] == [['a', 1]]
False
>>> [a] == [[1.0, 1]]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

The current behaviour is dependent upon having a list of numeric types.

Answer 2

The mistake was that I used a numpy.float32 instead of the normal float datatype as the values in my list. It was indeed a list, but I didn't check the datatype of the floats inside the list. If you want to recreate the error, you can do it like this

a = np.float32(1)
b = np.float32(2)
c = np.float32(3)
d = [[a,b],c]
e = [b,c]
d == e