Dataframe Warning : SettingWithCopyWarning in python

Question

Processing file from
http://portal.amfiindia.com/spages/NAV0.txt
to get output as follows:
31012017,1,1,135765,12,10.8536000,
31012017,1,1,135762,12,10.8543000,
31012017,1,1,135760,12,10.6599000,
31012017,1,1,135759,12,10.6554000,
31012017,1,1,135763,12,10.8536000,
..
..
..

I have tried using below code but getting below warning.

CODE:

import pandas
import numpy as np

#Sample file for NAV0.txt can be downloaded from url: http://portal.amfiindia.com/spages/NAV0.txt
#creating pandas with selected columns
df=pandas.read_table('NAV0.txt',sep=';',usecols=['Date','Scheme Code','Net Asset Value'])

#converting column with name 'Scheme Code' to digit to remove string part
fil_df=df[df['Scheme Code'].apply(lambda x : str(x).isdigit())]

#converting column with name 'Net Asset value' to numberic and set each value with 7 decimal places 
fil_df['Net Asset Value']=pandas.to_numeric(fil_df['Net Asset Value'],errors='coerce')
fil_df['Net Asset Value']=fil_df['Net Asset Value'].map(lambda x: '%2.7f' % x)


#Formating Date column as YYYMMDD
fil_df['Date']=pandas.to_datetime(fil_df['Date']).dt.strftime('%d%m%Y')

#adding extra column in dataframe
fil_df['ser1']=1
fil_df['ser2']=1
fil_df['period']=12
fil_df['lcol']=''
fil_df=fil_df[['Date','ser1','ser2','Scheme Code','period','Net Asset Value','lcol']]

#Converting datafile to csv
fil_df.to_csv('NAV_1.csv',index=False,header=None)
fil_df.dtypes

ERROR:

c:\users\administrator\appdata\local\programs\python\python35-32\lib\site-packages\ipykernel__main__.py:12: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

c:\users\administrator\appdata\local\programs\python\python35-32\lib\site-packages\ipykernel__main__.py:13: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

c:\users\administrator\appdata\local\programs\python\python35-32\lib\site-packages\ipykernel__main__.py:17: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

c:\users\administrator\appdata\local\programs\python\python35-32\lib\site-packages\ipykernel__main__.py:20: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

c:\users\administrator\appdata\local\programs\python\python35-32\lib\site-packages\ipykernel__main__.py:21: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

c:\users\administrator\appdata\local\programs\python\python35-32\lib\site-packages\ipykernel__main__.py:22: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

c:\users\administrator\appdata\local\programs\python\python35-32\lib\site-packages\ipykernel__main__.py:23: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

Csv file is getting generated as expected but how can I overcome this warning? I have tried using
fil_df.loc[ pandas.to_numeric(fil_df['Net Asset Value'],errors='coerce').map(lambda x: '%2.7f' % x]
but it didnt help.
Help would be appreciated.

I think you need `copy` - `fil_df=df[df['Scheme Code'].apply(lambda x : str(x).isdigit())].copy()` — jezrael, Apr 12 '17 at 10:57
I am not getting warning at above line. warning coming from this line : `fil_df['Net Asset Value']=pandas.to_numeric(fil_df['Net Asset Value'],errors='coerce')` — John, Apr 12 '17 at 11:11
@jezrael Thanks! it works.. last time i placed it at another line. — John, Apr 12 '17 at 11:15

score 1 · Accepted Answer · answered Apr 12 '17 at 11:17

1

I think you need add copy:

fil_df=df[df['Scheme Code'].apply(lambda x : str(x).isdigit())].copy()

If you modify values in fil_df later you will find that the modifications do not propagate back to the original data (df), and that Pandas does warning.

answered Apr 12 '17 at 11:17

jezrael

822,522
95
1,334
1,252

score 1 · Answer 2 · answered Jun 04 '18 at 03:07

1

If you know what your code is doing, you can use

pd.options.mode.chained_assignment = None  # default='warn'

in your code to disable this warning.

answered Jun 04 '18 at 03:07

Statham

4,000
2
32
45

score 0 · Answer 3 · edited May 23 '17 at 12:09

0

You'll get to the heart of the matter in adding new columns to a DataFrame from this guy's 2017 edit to this answer. Basically the route is to use the .assign('newCol' = enumerableValues )

edited May 23 '17 at 12:09

Community

1
1

answered Apr 13 '17 at 19:37

jxramos

7,356
6
57
105

Dataframe Warning : SettingWithCopyWarning in python

3 Answers3