2

Something strange is happening here (strange to me at least). I am trying to pop 'nan' values from one list and the corresponding values from another list. I was informed that you cannot pop from a list that is being enumerated, so I made a temporary list.

import numpy as np
a=[]
b=[]

for i in range(0,100): #alternating list of ints and nan
    if i/2. == i/2:
        a.append(i)
    else: 
        a.append(np.mean([])) #the only way I know how to make nan
    b.append(i*100)

atemp=a
print(len(a),len(atemp))
for i,v in enumerate(atemp):
    if np.isnan(v):
        a.pop(i)
        b.pop(i)
print(len(a),len(atemp))

A couple things are happening that I don't understand:

1) it appears to be popping nans from atemp AND a, even though I am only telling it a.pop(i) and not atemp.pop(i). Is there some sort of function overloading or something equating a and atemp that I don't see? How do I make them entirely separate lists (i.e. only pop from a and not atemp?

2) it is not popping all nan's and the for loop has to be repeated multiple times to pop all nan's

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Wes
  • 69
  • 6
  • 2
    `atemp = a` makes `atemp` and `a` point to the same list. If you want them to be distinct (i.e., remove from `atemp` does not remove from `a`), you need `atemp = a[:]`. – Two-Bit Alchemist Oct 16 '15 at 20:57
  • Aside: if you're using numpy, you can use `np.nan` directly to get a NaN value. Even without it, you can do `float("nan")`. – DSM Oct 16 '15 at 20:57
  • Thank you for the very fast and helpful replies. Unfortunately I get False with a[1]==np.nan and a[1]=float("nan") I am still not getting what I want. By popping from list a while enumerating list atemp I am running into a "pop index out of range" error. There has to be an easier way... perhaps using zip(a,b) – Wes Oct 16 '15 at 21:06
  • Yepp, see my answer. Using pop makes sense if you need the value that is popped and you need to update the list in place. – rebeling Oct 16 '15 at 21:07
  • @Wes: remember that `nan` isn't equal to itself, so x == nan will never be True. I was just pointing out you don't need to take the mean of an empty list to get a nan. – DSM Oct 16 '15 at 21:14
  • `atemp = a[:]` you are creating a reference – Padraic Cunningham Oct 16 '15 at 21:18
  • Possible duplicate of [How to clone or copy a list in Python?](http://stackoverflow.com/questions/2612802/how-to-clone-or-copy-a-list-in-python) – Chad S. Oct 16 '15 at 21:22
  • Thanks DSM. The fact that nan isn't equal to itself is now the crux of the problem for me. How do I test if the value is nan then? I have a[1]=nan. But a[1] == np.nan gives False and a[1] != np.nan gives True. These are the exact same results I get for a[2], which btw a[2]=2. I am failing on the Boolean part of this and I don't understand why. – Wes Oct 16 '15 at 21:28

3 Answers3

2

There are much simpler ways but if you want to use your own code, you need to copy a not just create a reference and use different logic to remove elements from your list, looping over the range len a -1 in reverse will give you the output you want:

import numpy as np

a = []
b = []

for i in range(0, 100):  # alternating list of ints and nan
    if i % 2 == 0:
        a.append(i)
    else:
        a.append(np.nan)  #the only way I know how to make nan
    b.append(i * 100)

atemp = a[:]
print(len(a), len(atemp))

for i in range(len(a)-1,-1,-1):
    if np.isnan(a[i]):
        a.pop(i)
        b.pop(i)
print(len(a), len(atemp))
Padraic Cunningham
  • 176,452
  • 29
  • 245
  • 321
1

What about this

# use this to filter None values in b
a, b = zip(*((x,y) for x,y in zip(a, b) if y))

# use this to filter np.nan in b
a, b = zip(*((x,y) for x,y in zip(a, b) if not np.isnan(y)))
rebeling
  • 718
  • 9
  • 31
  • *if y* proofs if y is something else then empty or None replace it like this: if y != np.nan or if not np.isnan(y) – rebeling Oct 16 '15 at 21:18
  • a[1]=nan is missing np, but u r right. Use np.isnan(x) instead, np.nan is not just a value its an object with a reference. When you compare it with another one then False make sense. But if you check the type you get it ;) – rebeling Oct 16 '15 at 21:30
1

This did it:

a, b = zip(*[(x,y) for x,y in zip(a, b) if np.isfinite(y)])

Thanks everyone. Sorry rebeling's answer got voted down as it helped me quite a bit.

Wes
  • 69
  • 6