How to deal with “SettingWithCopyWarning” in a for loop if statement

Question

Let's say column A is time-based, column B is salary.

I am using an if statement within a for loop trying to find "all salaries that are less than the previous one BUT ALSO greater than the following one." Then assign a new value ('YES') to another column (column C) of the rows that fulfill the condition. Finally, I want to grab the first column A that fulfill the above conditions.

The dataframe looks like this:

In [1]:
df = pd.DataFrame({'A':['2007q3','2007q4','2008q1','2008q2','2008q3','2008q4','2009q1','2009q2','2009q3'], 
                   'B':[14938, 14991, 14899, 14963, 14891, 14577, 14375, 14355, 14402]})
df['C'] = pd.Series()
df

Out [1]:
      A       B      C
0   2007q3  14938   NaN
1   2007q4  14991   NaN
2   2008q1  14899   NaN
3   2008q2  14963   NaN
4   2008q3  14891   NaN
5   2008q4  14577   NaN
6   2009q1  14375   NaN
7   2009q2  14355   NaN
8   2009q3  14402   NaN

The following code does the work but is showing the "SettingWithCopyWarning" warning, I am not sure which parts of the code is causing the problem...

In [2]:
for i in range(1, len(df)-1):
    if (df['B'][i] < df['B'][i-1]) & (df['B'][i] > df['B'][i+1]):
       df['C'][i] = 'YES'
df

Out [2]: 
      A       B      C
0   2007q3  14938   NaN
1   2007q4  14991   NaN
2   2008q1  14899   NaN
3   2008q2  14963   NaN
4   2008q3  14891   YES
5   2008q4  14577   YES
6   2009q1  14375   YES
7   2009q2  14355   NaN
8   2009q3  14402   NaN

In [3]: df['A'][df['C'] == 'YES'].iloc[0]
Out [3]:'2008q3'

Or maybe there's a better way to have the job done. Thank you!!!

Ben.T · Answer 1 · 2019-04-09T16:50:53.640

For more details on why you got SettingWithCopyWarning, I would suggest you to read this answer. It is mostly because selecting the column df['C'] and then selecting the row with [i] does a "chained assignment" that is flagged this way when you do df['C'][i] = 'YES'

For what you try to do, you can use np.where and shift on the column B such as:

import numpy as np
df['C'] = np.where((df.B < df.B.shift()) & (df.B > df.B.shift(-1)), 'YES', np.nan)

and you get the same output.

How to deal with “SettingWithCopyWarning” in a for loop if statement

1 Answers1