Can I replace values in a list conditionally?

Question

I am attempting to create a computed field in pandas, and I was given two formulas. Due to data issues, when one formula would result in a nan, I should be using the other. However, I cannot seem to get the function to output what I need.

This is what the data looks like:

        Nest    year    Hatched Unhatched   Pipped  Rel_Eggs
0        1      1980    120.0   NaN         NaN         120.0   
1        2      1980    NaN     NaN         NaN         NaN 
2        3      1980    62.0    NaN         NaN         117.0   
3        4      1980    0.0     NaN         NaN         NaN 
4        5      1980    0.0     NaN         NaN         NaN 
5        6      1980    110.0   NaN         NaN         114.0

Here is what I've already tried:

def new_success(data):
    succ = []
    for i in data.index:
        nval = data.loc[i]['Hatched']/(data.loc[i]['Hatched'] + data.loc[i]['Unhatched'] + data.loc[i]['Pipped'])
        if nval != np.inf and nval != np.nan:
                succ.append(nval)

        else:
            val = data.loc[i]['Hatched']/data.loc[i]['Rel_Eggs']
            succ.append(val)

    return succ

The first formula is working; the second, conditional bit is not. Based on the above dataframe, for these first 6 values I should be be getting

[1.0, nan, 0.523, nan, nan, 0.965]

but I just get

[nan, nan, nan, nan, nan, nan]

Try adding a `print(data.loc[i])` inside that loop to make sure that the data looks as you expect... — Samwise, Nov 03 '19 at 17:53
Rather, check the value of `nval`. It might be a simple `float` NaN, rather than a `numpy` value. — chepner, Nov 03 '19 at 18:00
Hm, yeah it doesn't even print if I put that in the second conditional. So I'm definitely missing something — chemmy, Nov 03 '19 at 18:00
@chepner you actually are right; this works if changed to ```if nval != np.inf and not math.isnan(nval):``` — chemmy, Nov 03 '19 at 18:04

FObersteiner · Accepted Answer · 2019-11-03T18:20:45.330

the unexpected result you get is likely related to this line: if nval != np.inf and nval != np.nan: - this is not how you check if a value is NaN. example:

import numpy as np
noval = np.nan
print(noval != np.nan)
# prints True - but one might expect False...

# however this works for inf:
noval = np.inf
print(noval != np.inf)
# prints False

use if np.isfinite(nval): instead.

novals = [np.nan, np.inf]
[print(np.isfinite(v)) for v in novals]
# False
# False

more info e.g. here.

Can I replace values in a list conditionally?

1 Answers1