Pandas: Change a specific column name in dataframe having multilevel columns

Question

I want to find the way change name of specific column in a multilevel dataframe.

With this data:

data = {
    ('A', '1', 'I'): [1, 2, 3, 4, 5], 
    ('B', '2', 'II'): [1, 2, 3, 4, 5], 
    ('C', '3', 'I'): [1, 2, 3, 4, 5], 
    ('D', '4', 'II'): [1, 2, 3, 4, 5], 
    ('E', '5', 'III'): [1, 2, 3, 4, 5], 
}

dataDF = pd.DataFrame(data)

This code not working:

dataDF.rename(columns = {('A', '1', 'I'):('Z', '100', 'Z')}, inplace=True)

Result:

    A   B   C   D   E
    1   2   3   4   5
    I   II  I   II  III
0   1   1   1   1   1
1   2   2   2   2   2
2   3   3   3   3   3
3   4   4   4   4   4
4   5   5   5   5   5

And also not:

dataDF.columns.values[0] = ('Z', '100', 'Z')

Result:

    A   B   C   D   E
    1   2   3   4   5
    I   II  I   II  III
0   1   1   1   1   1
1   2   2   2   2   2
2   3   3   3   3   3
3   4   4   4   4   4
4   5   5   5   5   5

But with combination of above codes working!!!

dataDF.columns.values[0] = ('Z', '100', 'Z')
dataDF.rename(columns = {('A', '1', 'I'):('Z', '100', 'Z')}, inplace=True)
dataDF

Result:

    Z   B   C   D   E
    100 2   3   4   5
    Z   II  I   II  III
0   1   1   1   1   1
1   2   2   2   2   2
2   3   3   3   3   3
3   4   4   4   4   4
4   5   5   5   5   5

Is this bug of Pandas?

score 29 · Accepted Answer · edited Nov 20 '21 at 02:43

This is my theory

pandas does not want pd.Indexs to be mutable. We can see this if we try to change the first element of the index ourselves

dataDF.columns[0] = ('Z', '100', 'Z')

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-32-2c0b76762235> in <module>()
----> 1 dataDF.columns[0] = ('Z', '100', 'Z')

//anaconda/envs/3.5/lib/python3.5/site-packages/pandas/indexes/base.py in __setitem__(self, key, value)
   1372 
   1373     def __setitem__(self, key, value):
-> 1374         raise TypeError("Index does not support mutable operations")
   1375 
   1376     def __getitem__(self, key):

TypeError: Index does not support mutable operations

But pandas can't control what you do the values attribute.

dataDF.columns.values[0] = ('Z', '100', 'Z')

we see that dataDF.columns looks the same, but dataDF.columns.values clearly reflects the change. Unfortunately, df.columns.values isn't what shows up on the display of the dataframe.

On the other hand, this really does seem like it should work. The fact that it doesn't feels wrong to me.

dataDF.rename(columns={('A', '1', 'I'): ('Z', '100', 'Z')}, inplace=True)

I believe the reason this only works after having changed the values, is that rename is forcing the reconstruction of the columns by looking at the values. Since we change the values, it now works. This is exceptionally kludgy and I don't recommend building a process that relies on this.

my recommendation

identify location of column name you want to change
assign name of column to the array of values
build new columns from scratch, explicity

from_col = ('A', '1', 'I')
to_col = ('Z', '100', 'Z')
colloc = dataDF.columns.get_loc(from_col)
cvals = dataDF.columns.values
cvals[colloc] = to_col

dataDF.columns = pd.MultiIndex.from_tuples(cvals.tolist())

dataDF

`dataDF.columns.values[0] = ('Z', '100', 'Z')` works for me. Printing `dataDF.columns` afterwards correctly reflects the change. What am I missing here? Did pandas fix this in a newer version? — Jon, Jun 09 '21 at 19:01

score 1 · Answer 2 · answered Nov 07 '16 at 07:24

1

You can simply change it like DF.columns.levels=[[u'Z', u'B', u'C', u'D', u'E'],[u'5', u'2', u'3', u'4', u'5'],[u'IIIIII', u'II', u'III']]

answered Nov 07 '16 at 07:24

Dark Matter

190
2
13

still I am not sure if its bug as your saying. – Dark Matter Nov 07 '16 at 07:25

novastar · Answer 3 · 2018-03-23T21:33:31.283

I came across this question as I was myself trying to find the solution for renaming the column names in a data frame with multiple levels. I tried the solution provided by @Dark Matter since it appeared to be very simple solution:

dataDF.columns.levels = [[u'Z', u'B', u'C', u'D', u'E'], [u'100', u'2', u'3', u'4', u'5'], [u'Z', u'II', u'III']]

But an error message was displayed:

C:\anaconda3\lib\site-packages\ipykernel_launcher.py:1: FutureWarning: setting `levels` directly is deprecated. Use set_levels instead
  """Entry point for launching an IPython kernel.

It appears that it worked but does not work anymore. So I used:

dataDF.columns.set_levels([['Z', 'B', 'C', 'D', 'E'],
                           ['100', '2', '3', '4', '5'],
                           ['Z', 'II', 'III']],
                          [0, 1, 2], inplace=True)

Result: dataDF

Z   B   C   D   E
100 2   3   4   5
Z   II  Z   II  III
0   1   1   1   1   1
1   2   2   2   2   2
2   3   3   3   3   3
3   4   4   4   4   4
4   5   5   5   5   5

Pandas: Change a specific column name in dataframe having multilevel columns

3 Answers3

Linked