Finding the index of items cotaining 'inf' or 'nan'

Question

The following is a sample of 1 item in my list:

array([[  1,   2,   3,
          43,   83,   92],
       [  12,   54,   93,
          23,   94,   83],
       [  23,   inf,   inf,
          inf,   inf,   inf],
       [  83,   33,   33,
          83,   13,   83],
       [  83,   nan,   83,
          73,   43,   43],
       [  43,   83,   93,
          22,   83,   54],
       [  66,   nan,   74,
          84,   84,   75],
       [  74,   44,   65,
          6,   9,   7],
       [  54,   9,   74,
          754,   55,   74]])

Some of those items contain inf or nan values. I'm thus trying to return the index of the items that contain such values. I thus tried doing the following:

for x in f:
    if float('inf') in x:
        idx.append(f.index(x))
    elif float('nan') in x:
        idx.append(f.index(x))

When I run my script however, I get the following error:

idx.append(f.index(x))
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Why is that? How can I solve this issue?

As a reminder, the above sample is only 1 item. Since it contains inf and nan, I would like to return the index of that item. The item is basically a list of lists.

Thanks.

This data does not have any infinity or nans. Can you please add a sample with expected output? You can use `np.isfinite`. — cs95, Feb 03 '18 at 00:59
what is f? what is idx? If I am not mistaken, the error as nothing to do with inf or nan... https://stackoverflow.com/questions/12647471/the-truth-value-of-an-array-with-more-than-one-element-is-ambigous-when-trying-t — Astrom, Feb 03 '18 at 01:06
@cᴏʟᴅsᴘᴇᴇᴅ I made an edit on my data to show a sample — Simplicity, Feb 03 '18 at 01:20
@Astrom "f" is simple the name of the list that contains the data, and "idx" is a list where I would like to add the indexes of items containing "inf" or "nan" — Simplicity, Feb 03 '18 at 01:21
The accepted answer isn't great, because you don't need a list comprehension at all. You can 100% vectorize this. But I can't write an answer if you don't show your expected output. — cs95, Feb 03 '18 at 01:47

score 2 · Accepted Answer · answered Feb 03 '18 at 01:27

Following suggestion by @coldspeed you can obtain an index of elements containing infs or nans as

idx = [i for i, arr in enumerate(f) if not np.isfinite(arr).all()]

There is a gotcha coming from idx.append(f.index(x)) that results in your error. It will return True if x is the first element of f and throw an error otherwise. It will also throw an error if you set f[0] = x.copy() and attempt to check x in f. Both are due to the fact that bool(x == x) is not well-defined.

score 1 · Answer 2 · answered Feb 03 '18 at 03:53

something that prints out like your Item in Python would be expected to be a numpy array

and can be most efficiently procesed in numpy as mentioned by cᴏʟᴅsᴘᴇᴇᴅ

to make the printout 'array' inf, nan into the np objects, and the 'array' into a np.array:

import numpy as np
inf, nan = np.inf, np.nan  

a = np.array([[  1,   2,   3,
          43,   83,   92],
       [  12,   54,   93,
          23,   94,   83],
       [  23,   inf,   inf,
          inf,   inf,   inf],
       [  83,   33,   33,
          83,   13,   83],
       [  83,   nan,   83,
          73,   43,   43],
       [  43,   83,   93,
          22,   83,   54],
       [  66,   nan,   74,
          84,   84,   75],
       [  74,   44,   65,
          6,   9,   7],
       [  54,   9,   74,
          754,   55,   74]])

then there are numpy functions just for these kind of operations

a_infs = np.where(np.isinf(a))

a_infs
Out[170]: (array([2, 2, 2, 2, 2], dtype=int64), array([1, 2, 3, 4, 5], dtype=int64))

a[a_infs]
Out[171]: array([ inf,  inf,  inf,  inf,  inf])

a_nans = np.where(np.isnan(a))

a_nans
Out[173]: (array([4, 6], dtype=int64), array([1, 1], dtype=int64))

a[a_nans]
Out[174]: array([ nan,  nan])

the two ca be combined

a_bads = tuple(np.hstack((a_infs, a_nans)))

a[a_bads]
Out[184]: array([ inf,  inf,  inf,  inf,  inf,  nan,  nan])

a_bads
Out[185]: 
(array([2, 2, 2, 2, 2, 4, 6], dtype=int64),
 array([1, 2, 3, 4, 5, 1, 1], dtype=int64))

or be converted to list and manipulated

a_infs = np.array(a_infs).tolist()

a_infs    
Out[175]: [[2, 2, 2, 2, 2], [1, 2, 3, 4, 5]]

[*zip(*a_infs)]
Out[176]: [(2, 1), (2, 2), (2, 3), (2, 4), (2, 5)]

Finding the index of items cotaining 'inf' or 'nan'

2 Answers2

Linked