Anaconda: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame

Question

I tried to run the following code:

#%% add offsests and remove not qualified data 
df=df[df['surfHeightr1_qual']==0] # eliminate values where the quality flag == 1

#add offset to SIN SurfType 1 and 3 --> this is needed when data is baseline C 
if df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==1) & (df['BaselineID']=='C')].any() == True:
df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==1)& (df['BaselineID']=='C')]=df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==1)& (df['BaselineID']=='C')]+59.959
if df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==3)& (df['BaselineID']=='C')].any() == True:   
df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==3)& (df['BaselineID']=='C')]=df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==3)& (df['BaselineID']=='C')]+59.959

in Python, but I get the folowing error:

SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==3)& (df['BaselineID']=='C')]=df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==3)& (df['BaselineID']=='C')]+59.959

Interestingly the error occur only at the second if, so I don't understand why the error occur. Also all following usages of dfis ignored by the code.

I use Python 3.6.3 with Anaconda in Ubuntui 16.04.

Does someone has an idea?

Thank you very much in advacne!

jezrael · Accepted Answer · 2018-06-06T13:12:40.357

I think need loc for avoid SettingWithCopyWarning, also code should be simplify by +=:

mask = (df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==3) & (df['BaselineID']=='C')
df.loc[mask, 'surfHeightr1'] +=59.959

EDIT:

For completely solution use numpy.select and for better readable each condition assign to separate variable:

df = pd.DataFrame({'BaselineID':list('CCCCCD'),
                   'surfHeightr1_qual':[0,0,0,0,0,1],
                   'OPERATION_MODE':['SIR_SIN_L2'] * 6,
                   'surfType':[1,3,1,5,1,0],
                   'surfHeightr1':[5,3,6,9,2,4],
                   'F':list('aaabbb')})

print (df)
  BaselineID  surfHeightr1_qual OPERATION_MODE  surfType  surfHeightr1  F
0          C                  0     SIR_SIN_L2         1             5  a
1          C                  0     SIR_SIN_L2         3             3  a
2          C                  0     SIR_SIN_L2         1             6  a
3          C                  0     SIR_SIN_L2         5             9  b
4          C                  0     SIR_SIN_L2         1             2  b
5          D                  1     SIR_SIN_L2         0             4  b

df=df[df['surfHeightr1_qual']==0].copy() # eliminate values where the quality flag == 1

#add offset to SIN SurfType 1 and 3 --> this is needed when data is baseline C 
m1 = df['OPERATION_MODE']=='SIR_SIN_L2'
m2 = df['BaselineID']=='C'
m3 = df['surfType']==1
m4 = df['surfType']==3

m123 = m1 & m2 & m3
m124 = m1 & m2 & m4

df['surfHeightr1'] += np.select([m123, m124], [59.959, 59.959], default=0)
print (df)
  BaselineID  surfHeightr1_qual OPERATION_MODE  surfType  surfHeightr1  F
0          C                  0     SIR_SIN_L2         1        64.959  a
1          C                  0     SIR_SIN_L2         3        62.959  a
2          C                  0     SIR_SIN_L2         1        65.959  a
3          C                  0     SIR_SIN_L2         5         9.000  b
4          C                  0     SIR_SIN_L2         1        61.959  b

Thank you very much for your answer! How can I combine the `mask = ... ` command in the if? Would it be `if mask`? — Dennis, Jun 06 '18 at 11:50
Yes, the value is an offset which will be applied in both cases — Dennis, Jun 06 '18 at 11:55
At the moment I have the problem to include mask in my code, I tried it in the following way: `mask = (df['surfHeightr1'][(df['OPERATION_MODE']=='SIR_SIN_L2') & (df['surfType']==3)& (df['BaselineID']=='C')]) if df.loc[mask, 'surfHeightr1'].any() == True: df.loc[mask, 'surfHeightr1'] +=59.959 ` — Dennis, Jun 06 '18 at 12:12

Anaconda: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame

1 Answers1