pandas chained_assignment warning exception handling

Question

If you worked in python with pandas you already know the chained_assignment warning when working on slices of dataframes (as e.g. described here).

I found the option pandas.options.mode.chained_assignment which can be set to

None, ignoring the warning
"warn", printing a warning message
"raise", raising an exception

compare with documentation

I included a minimal example, triggering this warning within a try..except..else block for exception handling. I expect an exception to be triggered only with the setting pandas.options.mode.chained_assignment = "raise", as shown in Example 3 below.

In this minimal example, the behaviour is as expected, so that Example 2, with pandas.options.mode.chained_assignment = "warn" only causes a warning message to be printed, but no exception is raised.

However in a larger framework, I see an exception being raised even though the parameter is set to pandas.options.mode.chained_assignment = "warn", checked with a print before like in the minimal example (see Example 4)

Is there any other pandas parameter influencing the behavior of this warning message concerning exception raising?

Here is a minimal example of code, setting/printing the pd.options.mode.chained_assignment parameter and showing the behaviour in a try..catch..except block.

import pandas as pd

# set the chained_assignment option
pd.options.mode.chained_assignment = "raise" # raises exception in case of warning
pd.options.mode.chained_assignment = "warn"  # prints warning in case of warning, no exception is raised
pd.options.mode.chained_assignment = None    # no warning message and no exception is raised

print "pd.options.mode.chained_assignment :", pd.options.mode.chained_assignment

# create a default pandas dataframe with two columns A,B
df = pd.DataFrame({"A" : [0, 1, 2], "B" : [3, 4, 5]})

print df

# exctract a slice of the given pandas dataframe
df2 = df[df["A"] > 0]

# exception handling
try :
    # try to modify the slice, triggering the pandas warning
    df2["C"] = 2
except :
    print "EXCEPTION RAISED"
else :
    print "NO EXCEPTION"

print df2

Example 1 Setting pd.options.mode.chained_assignment = None results in the following output (no warning, no exception)

pd.options.mode.chained_assignment : None
   A  B
0  0  3
1  1  4
2  2  5
NO EXCEPTION
   A  B  C
1  1  4  2
2  2  5  2

Example 2 Setting pd.options.mode.chained_assignment = "warn" results in the following output (a warning is printed, but no exception)

pd.options.mode.chained_assignment : warn
   A  B
0  0  3
1  1  4
2  2  5
NO EXCEPTION
C:\Users\my.name\my\directory\test.py:14:SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  df2["C"] = 2
   A  B  C
1  1  4  2
2  2  5  2

Example 3 Setting pd.options.mode.chained_assignment = "raise" results in the following output (an exception is raised)

pd.options.mode.chained_assignment : raise
A  B
0  0  3
1  1  4
2  2  5
EXCEPTION RAISED
   A  B  C
1  1  4  2
2  2  5  2

Example 4 This is what I see in a larger framework with exactly the same test code. I do not set the chained pd.options.mode.chained_assignment parameter explictly, but I see it's set to "warn", even though an exception is raised

pd.options.mode.chained_assignment warn
   A  B
0  0  3
1  1  4
2  2  5
EXCEPTION RAISED
   A  B  C
1  1  4  2
2  2  5  2

score 14 · Accepted Answer · answered Nov 08 '17 at 15:51

After a long search, the "bad guy" was found. Another developer included the following lines in his module

import warnings
warnings.filterwarnings('error')

This turns warnings into exceptions. For more details see warnings package documentation

Hence my warnings were treated as exceptions, although the pandas option was set to "warn"

pandas chained_assignment warning exception handling

1 Answers1