47

I have a large block of code that is, at some point somewhere, generating a setting with copy warning in pandas (this problem).

I know how to fix the problem, but I can't find what line number it is! Is there a way to back out the line number (apart from brute force methods like debug-stepping or putting in multiple prints)? The only output I get is the below, which doesn't go up the stack to my code:

C:\Anaconda3\lib\site-packages\pandas\core\frame.py:2302: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame  **kwargs)
smci
  • 32,567
  • 20
  • 113
  • 146
tim654321
  • 2,218
  • 2
  • 15
  • 19
  • 1
    How large is the code? I mean can't you just scan the code for `=`? – EdChum May 22 '15 at 08:44
  • You could divide and conquer, comment out the last half of your code and repeat until the warning goes away then you'll find the line, you could have more warnings later on but it'll probably trigger a pattern for you to recognise. I'd look for things where you take a reference to some rows or a df column and then try to assign to it – EdChum May 22 '15 at 08:49
  • 2
    I'd say you use your IDE's debugger, set a breakpoint at the given line and look at the stack trace. If you don't have an IDE, read about Python's internal debugger at https://docs.python.org/2/library/pdb.html – Klaus D. May 22 '15 at 08:49
  • Thanks Klaus - your tip did it. Placing a breakpoint at line 1182 of pandas\core\generic.py (where the warning was generated from, which I found by full-text searching the pandas/core directory), and then moving up the stack got me to the line number I need. – tim654321 May 22 '15 at 09:44
  • Apparently pandas now tells you the line number in more recent versions (> .14 I think) – tim654321 Jan 21 '16 at 23:26
  • [This answer](http://stackoverflow.com/questions/34999877/how-can-i-figure-out-and-access-pandas-warning-subclasses) to my similar question may be helpful too. – HaPsantran Jan 26 '16 at 01:46

1 Answers1

70

Set pd.options.mode.chained_assignment = 'raise'

This will throw an exception pointing to the line which triggers SettingWithCopyError.

UPDATE: how to catch the error, and interrogate the stacktrace to get the actual offending lineno:

import pandas as pd
from inspect import currentframe, getframeinfo
from pandas.core.common import SettingWithCopyError

pd.options.mode.chained_assignment = 'raise'

df = pd.DataFrame({'a': [1, 2, 3], 'b': [4, 5, 6]})

df2 = df[df['a'] == 2]

try:
    df2['b'] = 'foo'
except SettingWithCopyError:
    print('handling..')
    frameinfo = getframeinfo(currentframe())
    print(frameinfo.lineno)
smci
  • 32,567
  • 20
  • 113
  • 146
Julius
  • 886
  • 8
  • 13
  • Can the error be raised but the code runs further? I mean only to show the number of lines of code, where the warning(s) is/are happening. – PiMathCLanguage Oct 03 '18 at 08:33
  • Yes, you just need to catch the error. As for the line number you can either add it manually using logging/print statements, or extract it using inspect module. Please see the update in the answer. – Julius Oct 03 '18 at 18:16
  • 3
    If you want to change it back after finding the line, you could use `pd.options.mode.chained_assignment = 'warn'` – Jondiedoop Apr 29 '19 at 12:52