3

I need to compare if two numpy arrays are equal to a desired precision ignoring nan values.
For example:

a = [1,nan,3,nan]  
b = [1,0.2,3,4.1]  

Should pass the test.
I have tried using numpy.all function but i understand that it expects two array identical and i need to have some tolerance because float values may differ a little.
How can i achieve this?

cs95
  • 379,657
  • 97
  • 704
  • 746

2 Answers2

3

Use np.allclose and np.isnan:

mask = ~(np.isnan(a) | np.isnan(b))
np.allclose(a[mask], b[mask])

This correctly handles +/- inf and allows for small differences. Absolute and relative tolerances can be specified as parameters to allclose.

Paul Panzer
  • 51,835
  • 3
  • 54
  • 99
1

Integer Arrays
Mask your arrays using np.isfinite and compare with np.array_equal:

def array_nan_equal(a, b):
    m = np.isfinite(a) & np.isfinite(b)
    return np.array_equal(a[m], b[m])

assert array_nan_equal(
     np.array([1, np.nan, 3, np.nan]), np.array([1, 2, 3, 4])
)
assert not array_nan_equal(
     np.array([1, 4, 3, np.nan]), np.array([1, 2, 3, 4])
)

Note that if you want to account for +/-inf, you can follow the cue in @Paul Panzer's answer and use m = ~(np.isnan(a) & np.isnan(b)) instead of np.isfinite.


Floating Point Arrays
For floats, you'd need to compare within a tolerance, so substitute np.array_equal with a call to np.allclose:

def array_nan_close(a, b):
    m = np.isfinite(a) & np.isfinite(b)
    return np.allclose(a[m], b[m])

assert array_nan_close(
     np.array([1.3, np.nan, 3.4, np.nan]), np.array([1.3000001, 2, 3.4, 4])
)
assert not array_nan_close(
     np.array([1.1, 4.0, 3.5, np.nan]), np.array([1, 2, 3, 4])
)
cs95
  • 379,657
  • 97
  • 704
  • 746
  • What exactly are you trying to achieve by that shape check? – Paul Panzer Mar 13 '18 at 08:23
  • @PaulPanzer look at what being tired makes you do.. unfortunately now it resembles your answer a bit too much, so if you're uncomfortable with it, please feel free to edit it out of my answer. Cheers. Also passing you a vote. – cs95 Mar 13 '18 at 12:03