3

How to select part of the original variable "frame" such that changing "frame2" will also change "frame"? The following does not work.

import pandas as pd
frame = pd.DataFrame([[1,2,3],[1,5,6], [7,8,9]])
frame2 = frame.loc[frame.loc[:,0] == 1]
frame2.loc[:,1] = -99

Thanks!

Lukáš Zavřel
  • 43
  • 6
  • 1
    you need adding the .copy() – BENY Jun 28 '19 at 15:10
  • Without `.copy()` as WeNYoBen recommends, both frame and frame2 are pointed to the same object in memory. When you use `.copy()` you are creating a copy of the dataframe referenced by the frame variable.`frame2 = frame.loc[frame.loc[:,0] == 1].copy()` is what you are looking for. – Scott Boston Jun 28 '19 at 15:12
  • I think they _want_ to modify both at the same time, which is terrible practice by the way. The API has been designed in such a way that this cannot be done. – cs95 Jun 28 '19 at 15:14

1 Answers1

2

The first point of importance is that loc (and by extension, iloc, at, and iat) will always return a copy.

If you want a view, you'll have to index frame via __getitem__. Now, even this isn't guaranteed to return a view or copy -- that is an implementation detail and it isn't easy to tell.

Between the following indexing operations,

frame2 = frame[frame.iloc[:,0] == 1]
frame3 = frame[frame > 0]
frame4 = frame2[[0, 1]]

frame2._is_view
# False
frame3._is_view
# True
frame4._is_view
# False

only frame3 will be a view. The specifics also depend on dtypes and other factors (such as the shape of the slice), but this is an obvious distinction.

Despite frame3 being a view, modifications to it may either work or not, but they will never result in a change to frame. The devs have put a lot of checks in place (most notably the SettingWithCopyWarning) to prevent unintended side effects arising from modifying views.

frame3.iloc[:, 1] = 12345
frame3
   0      1  2
0  1  12345  3
1  1  12345  6
2  7  12345  9

frame
   0  1  2
0  1  2  3
1  1  5  6
2  7  8  9

TLDR; please look for a different way to do whatever it is you're trying to do.

cs95
  • 379,657
  • 97
  • 704
  • 746
  • 1
    Did not know about `_is_view`. +1 – harvpan Jun 28 '19 at 15:39
  • 1
    @harvpan I usually refrain from touching hidden or private attributes especially because they aren't always accurate and are liable to change or complete removal but in this case is a [common method](https://stackoverflow.com/questions/26879073/checking-whether-data-frame-is-copy-or-view-in-pandas) to check for views. – cs95 Jun 28 '19 at 15:41