1

I have a column in a DataFrame, which I want to change the values of. Calling the DataFrame df, and supposing one column is labelled A, I'm doing:

df.A = df.A.map(lambda x: -x)

This results in the operation actually being done, however it also results in the following Warning:

SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self[name] = value

OK, then I try with what is suggested in the Warning, and write:

df.loc[:,'A'] = df.A.map(lambda x: -x)

Only to get the exact same warning

SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self.obj[item] = s

OK, the last line is slightly different, but I don't know how to interpret it. Why am I getting the SettingWithCopyWarning even when I'm using .loc, as requested?

Then, I've tried a third method:

  • create a new column with the data
  • drop the old column
  • rename the new column with the old name

``

df['new'] = df.A.map(lambda x: -x)
df.drop('A',axis=1,inplace=True)
df.rename(columns={'new':'A'},inplace=True)

And again! I still get the warning:

SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  return super().rename(

At this point, I don't understand what is the proper way to replace a column with other values? Am I missing some important aspect of the framework? I just shouldn't try to use an existing column name for other data?

Michele Ancis
  • 1,265
  • 4
  • 16
  • 29
  • What about `df['A'] = df['A'].map(lambda x: -x)`? or just `df['A'] = -1 * df['A']` – Dani Mesejo Nov 30 '20 at 22:09
  • You should add `df = df.copy()` **before** `df['new'] = ...` when you define your dataframe. Also, I think `df['A'] = -df['A']` is a lot better, although you still need `copy()`. – Quang Hoang Nov 30 '20 at 22:12
  • 2
    IIUC, the issue is not with the current code but how this df was generated. Need to see that code to be sure. Also map is unnecessary, you can simply multiply the column by -1 – Vaishali Nov 30 '20 at 22:13
  • @DaniMesejo your proposal gives the exact same outcome :-/ - It changes the DF, but issues the same warning. @QuangHoang `df['A'] = - df['A']` gives the same warning - Why should I do `df = df.copy()`? – Michele Ancis Nov 30 '20 at 22:16
  • @Vaishali I think you might have pointed me to the right place, although I'm not sure how this work. The `df` I've proposed is fictional. The real one is generated by something like this: `df_master = pd.read_csv('file')` `df = df_master[df_master['column_x'] == 'val_x']` Is this the issue? Because I'm using `.loc` on a view of the original `df_master`? – Michele Ancis Nov 30 '20 at 22:21
  • I did `df['A'] = -df['A']` and I didn't get any warning msgs. Are you using the latest version of pandas and python pls? – Joe Ferndz Nov 30 '20 at 22:27
  • df = df_master[df_master['column_x'] == 'val_x'].copy() should solve the issue. Check out cs95's [answer](https://stackoverflow.com/questions/20625582/how-to-deal-with-settingwithcopywarning-in-pandas) that explains this really well. – Vaishali Nov 30 '20 at 22:44
  • @JoeFerndz `pandas` version is 1.0.5 - `python` version is 3.8.3 However, it looks like what's happening is tied to me not creating a `df.copy()` – Michele Ancis Dec 01 '20 at 20:02
  • I didnt have to create df.copy(). Interesting – Joe Ferndz Dec 01 '20 at 20:47
  • @JoeFerndz this has to do with trying to assign values to either `copy` or `view` of an object. I've read @cs95 answer, which explains *nearly* everything to me, but I'm still in need to think this through. I'll probably extend my question, to see whether I can solve my doubts. – Michele Ancis Dec 01 '20 at 21:20

0 Answers0