95

I have a pandas dataframe: data. it has columns ["name", 'A', 'B']

What I want to do (and works) is:

d2 = data[data['name'] == 'fred'] #This gives me multiple rows
d2['A'] = 0

This will set the column A on the fred rows to 0. I've also done:

indexes = d2.index
data['A'][indexes] = 0

However, both give me the same warning: 

/Users/brianp/work/cyan/venv/lib/python2.7/site-packages/pandas/core/indexing.py:128: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy

How does pandas WANT me to do this?

Brad Solomon
  • 38,521
  • 31
  • 149
  • 235
Brian Postow
  • 11,709
  • 17
  • 81
  • 125

1 Answers1

147

This is a very common warning from pandas. It means you are writing in a copy slice, not the original data so it might not apply to the original columns due to confusing chained assignment. Please read this post. It has detailed discussion on this SettingWithCopyWarning. In your case I think you can try 

data.loc[data['name'] == 'fred', 'A'] = 0
Community
  • 1
  • 1
Andreas Hsieh
  • 2,080
  • 1
  • 10
  • 8
  • 3
    I was about to post the same thing. A logical "one-liner" is better than unnecessary lines. – tnknepp Jun 15 '16 at 17:23
  • You're right. Thanks for reminding me. – Andreas Hsieh Jun 15 '16 at 17:29
  • 1
    Isn't there a straight forward way to return a new df with the columns edited, and leave the original unchanged? – user48956 Jul 05 '17 at 18:42
  • 17
    A lot of people say this is the correct way, and this is the way I go with as well. However, sometimes I get the warning anyway saying I'm setting values on a copy and advices me to use .loc when I'm already using. Anyone experienced the same thing? – Calvin Ku Oct 20 '17 at 03:34
  • 10
    @CalvinKu, yes! I am getting the same warning when doing what it is asking me to do! IMO, this is ambiguous behavior and should go down as a bug, but the Pandas people are tired of hearing about it, so I have little confidence it will be addressed... Such a shame... especially coming from R. – Bryan Goggin Oct 23 '17 at 21:04
  • 5
    It's interesting that sometimes I get this and it won't go away no matter how I refactor it. But then when I run the same code again some time later the warning is gone. I'm guessing the implementation of this part of pandas isn't very robust so you see false positives like this once in a while. But what bugs me is it seems like it doesn't happen to some people so they are convinced that it's your code that is wrong...haha – Calvin Ku Oct 26 '17 at 09:33
  • @CalvinKu, yep, I get the same warning using this solution. – branwen85 Dec 20 '17 at 12:04
  • 1
    @user48956 (very late to the party, but) you can do `data[:].loc[data['name'] == 'fred', 'A'] = 0` to get a copy. – JakeCowton Oct 02 '18 at 13:08
  • 3
    @CalvinKu, this happens when the dataframe you're assigning to is a view of another dataframe. E.g consider the code: {a = pd.DataFrame({'x':[1],'y':[1]}); b = a[['x']]; b.loc[:,'x'] = 0 }. Here you will get a settingwithcopy warning to inform you that you have changed the values of b, but not a. – Viktoriya Malyasova Oct 14 '19 at 08:25
  • @ViktoriyaMalyasova Thanks, this is the correct way, so I guess it's not a pandas bug. – Rishabh Gupta Feb 04 '23 at 16:36