Removing nan elements from matrix

Question

I have a bunch of matrices eq1, eq2, etc. defined like

from numpy import meshgrid, sqrt, arange
# from numpy import isnan, logical_not

xs = arange(-7.25, 7.25, 0.01)
ys = arange(-5, 5, 0.01)
x, y = meshgrid(xs, ys)

eq1 = ((x/7.0)**2.0*sqrt(abs(abs(x)-3.0)/(abs(x)-3.0))+(y/3.0)**2.0*sqrt(abs(y+3.0/7.0*sqrt(33.0))/(y+3.0/7.0*sqrt(33.0)))-1.0)

eq2 = (abs(x/2.0)-((3.0*sqrt(33.0)-7.0)/112.0)*x**2.0-3.0+sqrt(1-(abs(abs(x)-2.0)-1.0)**2.0)-y)

where eq1, eq2, eq3, etc. are large square matrices. As you can see, there are many nan elements surrounding a 'block' of plot-able values. I want to remove all the nan values whilst keeping the shape of the block of the valid values in the matrix. Note that these 'blocks' can be located anywhere in the eq1, eq2 matrix.

I've looked at answers given in Removing nan values from an array and Removing NaN elements from a matrix, but these don't seem to be completely relevant to my case.

"remove" as in take out `nan` and leave the non-`nan` values as they are in the matrix ... kind of like cutting off all the crusts when you're making a sandwich. — Medulla Oblongata, Jul 29 '14 at 23:25
`sqrt(abs(abs(x)-3.0)/(abs(x)-3.0))` - why would you do this? The values of the array inside the sqrt are always 1, -1, or NaN. — user2357112, Jul 29 '14 at 23:34
the matrices I get from inputting your sample code are not square — Gabriel, Jul 29 '14 at 23:48

score 3 · Accepted Answer · answered Jul 29 '14 at 23:36

IIUC, you can use boolean indexing with np.isnan to keep the slices. There are probably slicker ways to do this, but starting from something like:

>>> eq = np.zeros((5,6)) + np.nan
>>> eq[2:4, 1:3].flat = [1,np.nan,3,4]
>>> eq
array([[ nan,  nan,  nan,  nan,  nan,  nan],
       [ nan,  nan,  nan,  nan,  nan,  nan],
       [ nan,   1.,  nan,  nan,  nan,  nan],
       [ nan,   3.,   4.,  nan,  nan,  nan],
       [ nan,  nan,  nan,  nan,  nan,  nan]])

You could select the rows and columns with data using something like

>>> eq = eq[:,~np.isnan(eq).all(0)]
>>> eq = eq[~np.isnan(eq).all(1)]
>>> eq
array([[  1.,  nan],
       [  3.,   4.]])

score 2 · Answer 2 · answered Jul 29 '14 at 23:38

2

Short and sweet,

eq1_c = eq1[~np.isnan(eq1)]

np.isnan returns a bool array that can be used to index your original array. Take its negation and you will get back the non-nan values.

answered Jul 29 '14 at 23:38

Gabriel

10,524
1
23
28

1

Yes, I've tried that. I just get a single array. It doesn't preserve the shape of the matrix that contains all the non-`nan` values. – Medulla Oblongata Jul 29 '14 at 23:41
yes, i see. what do you want to happen if you're non-NaN values don't form a square matrix? Or in other words, what if your NaN values don't form a cross in the original matrix? More generally, this might be a question of how do you identify square blocks of `True` in a bool matrix – Gabriel Jul 29 '14 at 23:44

score 0 · Answer 3 · answered Jul 29 '14 at 23:35

One option is to manually iterate through the grid and check for Nan values. A Nan value is easy to spot because comparing it to itself will result in False. You could use this to set all Nan values to 0.0 for example.

for x in xrange(len(eq1)):
    for y in xrange(len(eq1[x])):
        v = eq1[x][y]
        if v!=v:
            eq1[x][y] = 0.0

Removing nan elements from matrix

3 Answers3