1

Is there any resolution in numpy with regards to the issue described in this SO post FutureWarning: elementwise comparison failed; returning scalar, but in the future will perform elementwise comparison. My numpy version is 1.19.1 and using python 3.8.5.

a = np.array(['aug', False, False, False])

a == 'aug'

array([ True, False, False, False])

But:

a == False

<ipython-input-88-f9ff25cfe387>:1: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  a == False

Same with np.nan:

a = array(['aug', np.nan, np.nan, np.nan])
a == 'aug'

array([ True, False, False, False])

But:

a == np.nan

<ipython-input-1236-9224919e9367>:1: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  a == np.nan

False
codingknob
  • 11,108
  • 25
  • 89
  • 126
  • Try `a[:,None,None] == 'False'` – hpaulj Nov 21 '20 at 19:32
  • 1
    `a` is string dtype. Why are you trying to test bool object, `False` or float object `nan` against that? The string comparison returns a boolean array of matching shape (why the extra dimensions that don't add anything to the issue?). The failed comparison returns a scalar False with the warning. Your `a` array is not equal to `False` or `nan`. – hpaulj Nov 21 '20 at 20:10
  • @hpaulj I cleaned up my original question. It is unclear why numpy cannot distinguish a boolean or a np.nan in an array that has a string element. – codingknob Nov 30 '20 at 18:56
  • You haven't bothered to look at your `a` arrays, have you? – hpaulj Nov 30 '20 at 19:40

1 Answers1

2

Look at the arrays after you create them:

In [58]: a = np.array(['aug', False, False, False])
    ...: 
In [59]: a
Out[59]: array(['aug', 'False', 'False', 'False'], dtype='<U5')
In [60]: a == 'aug'
Out[60]: array([ True, False, False, False])
In [61]: a == False
<ipython-input-61-f9ff25cfe387>:1: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  a == False
Out[61]: False
In [62]: a == 'False'
Out[62]: array([False,  True,  True,  True])

It's a string dtype array. Testing for matching strings works. Testing for a nonstring value is wrong.

Same for nan:

In [64]: a = np.array(['aug', np.nan, np.nan, np.nan])
In [65]: a
Out[65]: array(['aug', 'nan', 'nan', 'nan'], dtype='<U3')
In [66]: a == 'nan'
Out[66]: array([False,  True,  True,  True])

If you must mix types - string, boolean, float, you can specify object dtype. That makes the array more list-like.

In [67]: a = np.array(['aug', False, False, False], object)
In [68]: a
Out[68]: array(['aug', False, False, False], dtype=object)
In [69]: a == 'aug'
Out[69]: array([ True, False, False, False])
In [70]: a == False
Out[70]: array([False,  True,  True,  True])
In [71]: a == True
Out[71]: array([False, False, False, False])

But it doesn't help with np.nan. nan is a special kind of float that isn't equal to anything, not even itself.

In [72]: a = np.array(['aug', np.nan, np.nan, np.nan], object)
In [73]: a
Out[73]: array(['aug', nan, nan, nan], dtype=object)
In [74]: a=='aug'
Out[74]: array([ True, False, False, False])
In [75]: a == np.nan
Out[75]: array([False, False, False, False])

isnan is the correct way to test for nan, but it only works with float dtype arrays:

In [76]: np.isnan(a)
Traceback (most recent call last):
  File "<ipython-input-76-da86cb21b8a4>", line 1, in <module>
    np.isnan(a)
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • This answered all my questions. Thank you. So the nan is a special type that won't even work with a object dtype. Okay let me rethink my data and see if I can re-structure things to not mix dtypes. Appreciate your help on this. Ty. – codingknob Nov 30 '20 at 19:48