How do I select a row from a complex numpy array?

Question

I have read 10-20 different question/answers and cannot find an example like this. I want to select rows out of a numpy array as follows:

test = [ [ [0], np.zeros((250,250)), np.array([0,0]) ],
         [ [0], np.zeros((250,250)), np.array([1,1]) ],
         [ [1], np.zeros((250,250)), np.array([2,2]) ],
         [ [2], np.zeros((250,250)), np.array([2,2]) ],
         [ [2], np.zeros((250,250)), np.array([2,2]) ]
       ]

Now convert the list to an numpy array and print the first column:

nptest = np.array(test)
print (nptest[:,0])
> output is: [[0] [0] [1] [2] [2]]

Now try to select just the row where the first element is = 1

just_1s = nptest[nptest[:,0] == 1]
print (just_1s)
> output is []

I don't understand this output.

In my actual problem set I have 100s of each with an arbitrary number of rows with values of 0-15 in the first column. Using the example data above, the desired result would be three numpy arrays as follows:

just_0s = [[ [0], np.zeros((250,250)), np.array([0,0]) ],
           [ [0], np.zeros((250,250)), np.array([1,1]) ]
          ]

just_1s = [[ [1], np.zeros((250,250)), np.array([2,2]) ]]

just_2s = [[ [2], np.zeros((250,250)), np.array([2,2]) ],
           [ [2], np.zeros((250,250)), np.array([2,2]) ]
          ]

When I print `nptest[:,0]` I get `array([list([0]), list([0]), list([1]), list([2]), list([2])], dtype=object)`, None of those things equal 1, hence your empty array. — o-90, Jul 31 '17 at 21:30
So do I just have to do a for loop through the numpy array? It seems like there should be a way to select these rows based on the value in that column... — Brian F, Jul 31 '17 at 21:35

keepAlive · Accepted Answer · 2017-08-24T23:53:21.767

1

Use list comprehension

just_1s = [el for el in nptest if el[0] == [1]]

But there is actually no need to work with nd.arrays, using the original list is ok

just_1s = [el for el in test if el[0] == [1]]

If your point is why does one get [] when doing nptest[nptest[:, 0] == [1]] (note that I test [1] instead of 1), I do not know. Because according to me (and Sam Marinelli) it should have worked. We are wrong.

But if one looks closer, it appears that

just_1s = nptest[ nptest[:,0].tolist().index([1]) ]

works fine but only if [1] is unique. Which is not the case of, e.g. [2].

edited Aug 24 '17 at 23:53

answered Jul 31 '17 at 22:03

keepAlive

6,369
5
24
39

I left for a while, but I had already tried the [1] thought. I am going to try the use "list comprehension" approach and will come back to plus 1 if it does the trick. It looks like it will. I do want a numpy array at the end of the rainbow, but I can do that after I separate the list into its components. Back in a few... – Brian F Jul 31 '17 at 23:19
Thanks to all but, special thanks to Kanak. The list comprehension on the original list is the best of the solutions. Thanks for the rapid help and response! – Brian F Jul 31 '17 at 23:41

hpaulj · Answer 2 · 2017-07-31T23:10:15.343

This is list produces a (5,3) array of objects:

In [47]: nptest=np.array(test)
In [49]: nptest.shape
Out[49]: (5, 3)
In [50]: nptest.dtype
Out[50]: dtype('O')
In [51]: nptest[:,0]
Out[51]: array([list([0]), list([0]), list([1]), list([2]), list([2])], dtype=object)

The first column is an array (1d) of lists. In my newer numpy version that's more explicit.

Doing an equality test on an array, or even list, of lists is not easy. After trying several things I found this:

In [52]: arow = nptest[:,0]

In [56]: [x[0]==1 for x in arow]
Out[56]: [False, False, True, False, False]
In [57]: mask = [x[0]==1 for x in arow]
In [58]: nptest[mask,:]
Out[58]: 
array([[list([1]),
        array([[ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       ..., 
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.],
       [ 0.,  0.,  0., ...,  0.,  0.,  0.]]),
        array([2, 2])]], dtype=object)

Or we could turn the array of lists into an array of numbers:

In [60]: np.array(arow.tolist())
Out[60]: 
array([[0],
       [0],
       [1],
       [2],
       [2]])
In [61]: np.array(arow.tolist())==1
Out[61]: 
array([[False],
       [False],
       [ True],
       [False],
       [False]], dtype=bool)

Or test for [1] instead of 1. Lists match lists, not their contents.

In [64]: [x==[1] for x in arow]
Out[64]: [False, False, True, False, False]

This may be considered as an answer to my [question](https://stackoverflow.com/questions/45426587/what-is-going-on-behind-this-numpy-selection-behavior) ? — keepAlive, Jul 31 '17 at 23:08

score 0 · Answer 3 · answered Jul 31 '17 at 21:38

I believe the issue is with the expression nptest[:, 0] == 1. You've already shown that nptest[:, 0] returns [[0], [0], [1], [2], [2]] as expected. You can see that none of these values is equal to 1, so nptest[nptest[:, 0] == 1] will be empty. Try nptest[nptest[:, 0] == [1]] instead.

How do I select a row from a complex numpy array?

3 Answers3

Linked