0

I intended to drop all rows in a dataframe that I no longer need using the following:

df = df[my_selection]

where my_selection is a series of boolean values.

Later when I tried to add a column as follows:

df['New column'] = pd.Series(data)

I got the well-known "SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead"

Does this mean that df is actually a slice of its former self?

Or why am I being accused of assigning values to a slice?

Demo code:

import pandas as pd

data = {
    'A': pd.Series(range(8)),
    'B': pd.Series(range(8,0,-1))
}
df = pd.DataFrame(data)
df

Output: 

   A  B
0  0  8
1  1  7
2  2  6
3  3  5
4  4  4
5  5  3
6  6  2
7  7  1

This causes a warning:

my_selection = df['A'] < 4
df = df[my_selection]
df['C'] = pd.Series(range(4))

This does not create a warning:

df = pd.DataFrame(data)
df['C'] = pd.Series(range(8))

Should I be using df.drop?

Bill
  • 10,323
  • 10
  • 62
  • 85
  • 1
    Check out [SettingWithCopyWarning](https://stackoverflow.com/questions/20625582/how-to-deal-with-settingwithcopywarning-in-pandas) – Sociopath Feb 16 '18 at 05:47
  • 1
    Yes, df is a slice taken out of the original df. Just tell pandas you are aware of what you are doing by `df = df[my_selection].copy()` – ayhan Feb 16 '18 at 05:53
  • Thanks @Akshay. So it is 'false positive' warning then? Other than disabling this warning is there a more friendly way to avoid it? – Bill Feb 16 '18 at 05:54
  • @ayhan that sounds good but will it duplicate the dataframe in memory? Or will the old one get garbage collected? – Bill Feb 16 '18 at 05:55
  • 1
    That generates a copy either way. By using `df df[myselection].copy()` you are sending a signal that it is an explicit copy so of course the changes will not be reflected in the original one. You can achieve the same with `df.is_copy = None` after taking the slice. You are overwriting the original df and there is no name pointing to the original df anymore so yeah, it will be collected by garbage collector. – ayhan Feb 16 '18 at 06:44

0 Answers0