0

I have a Dataframe at hand. After making predictions, I want to update that dataframe with these values. I know there are similar questions, and answers around this site. I tried all of them. But I couldn't succeed. It gives the warning of "A value is trying to be set on a copy of a slice from a DataFrame." and do not update the original dataframe actually.

According to the questions similar to my problem, I tried changing the code with iloc, loc, iat etc. Nothing has worked.

My code takes a subdataframe from pred_set with filter, then makes predictions rowwise in order. Prediction of the upper row must update cells of lower rows. So that, i am making future predictions with moving months.

for key in list(mi.unique_everseen(pred_set['from'] + pred_set['to'])):
    for i in range(0, predmonths):
        pred = pred_set[(pred_set['from'] + pred_set['to']) == key].iloc[[i],:] 
        pred_val = all_pipe_final.predict(pred)
        for j in range (i+1, predmonths):
            pred_set.loc[(pred_set['from'] + pred_set['to']) == key, :].iloc[j, pred_set.columns.get_loc(str('M-'+str(j)))] = pred_val[0]
            #variation 2#pred_set[(pred_set['from'] + pred_set['to']) == key].iloc[j, pred_set.columns.get_loc(str('M-'+str(j)))] = pred_val[0]
            #variation 3#pred_set[(pred_set['from'] + pred_set['to']) == key].iloc[[j], [pred_set.columns.get_loc(str('M-'+str(j)))]] = pred_val[0]

As I stated before, this code part and variations do not update the cells. It looks like, i am working on a copy of dataframe, not the original one. What is the reason of that, I am wondering.

Eventually I changed the structure and added:

pred_subset = pred_set[(pred_set['from'] + pred_set['to']) == key]

I recorded calculations on this new dataframe, it worked. I don't know the reason.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Ali Kılınç
  • 11
  • 4

1 Answers1

0

When you get A value is trying to be set on a copy of a slice from a DataFrame., it means that you are working on a copy, so it means that what you change will also impact the "source data".

It is usually a good warning, which can be solved by using df = df.copy() ... if you really don't want a copy by setting pd.options.mode.chained_assignment to None (and restoring it after your problematic block, obviously)

Now, in your case, I'm suspecting the pred_set.loc[(pred_set['from'] + pred_set['to']) == key, :].iloc[....] to be the cause of that. The risk seems under control, so you can disable that warning in that block.

Here's an example:

import pandas as pd
import seaborn as sns
import warnings

iris = sns.load_dataset('iris')
irisSub = iris.head(3)

irisSub.species = 'Spam'  # will raise the warning
print(iris)  # iris top 3 rows are impacted

old_ca = pd.options.mode.chained_assignment
pd.options.mode.chained_assignment = None
irisSub.species = 'Eggs'  # won't raise the warning
pd.options.mode.chained_assignment = old_ca  # restore
print(iris)

See also How to deal with SettingWithCopyWarning in Pandas?.

Michael Hooreman
  • 582
  • 1
  • 5
  • 16