Stack Overflow
Stack Overflow

Questions tagged [chained-assignment]

Chaining of consecutive and separate index operations with Python-pandas objects.

Chained assignment (or chained indexing) as it relates to indexing is a chaining of consecutive and separate index operations, where successive index operations are done on a copy of a portion of the result from the first operation.

Assignment with chained indexing will generate a SettingWithCopyWarning and should be generally be avoided. However, there are a minority of instances where the warning can be safely ignored. For these situations consider using:

with pd.option_context('mode.chained_assignment', None):
    # Code inside this `with` block will not issue the warning

For more see:

46 questions
1354
votes
21 answers

How to deal with SettingWithCopyWarning in Pandas

Background I just upgraded my Pandas from 0.11 to 0.13.0rc1. Now, the application is popping out many new warnings. One of them like this: E:\FinReporter\FM_EXT.py:449: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a…
bigbug
  • 55,954
  • 42
  • 77
  • 96
1295
votes
32 answers

How to add a new column to an existing DataFrame?

I have the following indexed DataFrame with named columns and rows not- continuous numbers: a b c d 2 0.671399 0.101208 -0.181532 0.241273 3 0.446172 -0.243316 0.051767 1.577318 5 0.614758 0.075793 -0.451460…
tomasz74
  • 16,031
  • 10
  • 37
  • 51
354
votes
8 answers

why should I make a copy of a data frame in pandas

When selecting a sub dataframe from a parent dataframe, I noticed that some programmers make a copy of the data frame using the .copy() method. For example, X = my_dataframe[features_list].copy() ...instead of just X =…
Elizabeth Susan Joseph
  • 6,255
  • 7
  • 20
  • 23
350
votes
10 answers

Extracting specific selected columns to new DataFrame as a copy

I have a pandas DataFrame with 4 columns and I want to create a new DataFrame that only has three of the columns. This question is similar to: Extracting specific columns from a data frame but for pandas not R. The following code does not work,…
SpeedCoder5
  • 8,188
  • 6
  • 33
  • 34
189
votes
3 answers

What rules does Pandas use to generate a view vs a copy?

I'm confused about the rules Pandas uses when deciding that a selection from a dataframe is a copy of the original dataframe, or a view on the original. If I have, for example, df = pd.DataFrame(np.random.randn(8,8), columns=list('ABCDEFGH'),…
orome
  • 45,163
  • 57
  • 202
  • 418
111
votes
12 answers

Python: Pandas Dataframe how to multiply entire column with a scalar

How do I multiply each element of a given column of my dataframe with a scalar? (I have tried looking on SO, but cannot seem to find the right solution) Doing something like: df['quantity'] *= -1 # trying to multiply each row's quantity column with…
labheshr
  • 2,858
  • 5
  • 23
  • 34
95
votes
1 answer

Correct way to set value on a slice in pandas

I have a pandas dataframe: data. it has columns ["name", 'A', 'B'] What I want to do (and works) is: d2 = data[data['name'] == 'fred'] #This gives me multiple rows d2['A'] = 0 This will set the column A on the fred rows to 0. I've also…
Brian Postow
  • 11,709
  • 17
  • 81
  • 125
73
votes
3 answers

Checking whether data frame is copy or view in Pandas

Is there an easy way to check whether two data frames are different copies or views of the same underlying data that doesn't involve manipulations? I'm trying to get a grip on when each is generated, and given how idiosyncratic the rules seem to be,…
nick_eu
  • 3,541
  • 5
  • 24
  • 38
70
votes
4 answers

Pandas: Knowing when an operation affects the original dataframe

I love pandas and have been using it for years and feel pretty confident I have a good handle on how to subset dataframes and deal with views vs copies appropriately (though I use a lot of assertions to be sure). I also know that there have been…
ejolly
  • 809
  • 1
  • 8
  • 6
47
votes
1 answer

How to find the line that is generating a Pandas SettingWithCopyWarning?

I have a large block of code that is, at some point somewhere, generating a setting with copy warning in pandas (this problem). I know how to fix the problem, but I can't find what line number it is! Is there a way to back out the line number (apart…
tim654321
  • 2,218
  • 2
  • 15
  • 19
34
votes
2 answers

Confusion about pandas copy of slice of dataframe warning

I've looked through a bunch of questions and answers related to this issue, but I'm still finding that I'm getting this copy of slice warning in places where I don't expect it. Also, it's cropping up in code that was running fine for me previously,…
Sam Lilienfeld
  • 475
  • 1
  • 5
  • 15
31
votes
1 answer

Pandas: Subindexing dataframes: Copies vs views

Say I have a dataframe import pandas as pd import numpy as np foo = pd.DataFrame(np.random.random((10,5))) and I create another dataframe from a subset of my data: bar = foo.iloc[3:5,1:4] does bar hold a copy of those elements from foo? Is there…
Amelio Vazquez-Reina
  • 91,494
  • 132
  • 359
  • 564
27
votes
1 answer

Type hints and chained assignment and multiple assignments

I guess these two questions are related, so I'll post them together: 1.- Is it possible to put type hint in chained assignments? These two attempts failed: >>> def foo(a:int): ... b: int = c:int = a File "", line 2 b: int = c:int =…
steffen
  • 8,572
  • 11
  • 52
  • 90
25
votes
4 answers

why is blindly using df.copy() a bad idea to fix the SettingWithCopyWarning

There are countless questions about the dreaded SettingWithCopyWarning I've got a good handle on how it comes about. (Notice I said good, not great) It happens when a dataframe df is "attached" to another dataframe via an attribute stored in…
piRSquared
  • 285,575
  • 57
  • 475
  • 624
24
votes
2 answers

Pandas SettingWithCopyWarning

Python 3.4 and Pandas 0.15.0 df is a dataframe and col1 is a column. With the code below, I'm checking for the presence of the value 10 and replacing such values with 1000. df.col1[df.col1 == 10] = 1000 Here's another example. This time, I'm…
ba_ul
  • 2,049
  • 5
  • 22
  • 35
1
2 3 4