Add multiple columns at once to multiindex Pandas dataframe

Question

I have a dataframe multiindex pandas dataframe df

First    Foo     Bar
Second   Begin   Begin
1        5       1
2        4       4
3        6       6

And I want to add two columns of the same name

First    Foo             Bar
Second   Begin   End     Begin   End
1        5       1       1       2       
2        4       5       4       4       
3        6       7       6       7

From this source (new):

First    Foo    Bar
1        1      2
2        5      4
3        7      7

I tried things like df[:] = new[:] but this returned only NaN

An alternative would be to use something like a for-loop but that's not the Pandas approach. Searching the web did not give me any insights as to solving this problem.

How can I add new columns with the same name and shape to every first level of a multiindex Pandas dataframe?

Edit:

This approach df[('Foo', 'End')] = new['Foo'] df[('Bar', 'End')] = new['Bar'] is not an option because in my actual problem there is not two columns to be added, but hundreds of columns.

why are you using a multi-index? There may be an easier way to achieve what you want — anon01, Aug 24 '20 at 20:04
Does this answer your question? [Selecting columns from pandas MultiIndex](https://stackoverflow.com/questions/18470323/selecting-columns-from-pandas-multiindex) — Trenton McKinney, Aug 24 '20 at 20:25
@TrentonMcKinney no my question is not about selecting but about inserting columns — Duco, Aug 24 '20 at 21:00
@TrentonMcKinney yes, but I don't believe this is the 'Pandian' way. Using for loops breaks the logic of Pandas. — Duco, Aug 25 '20 at 06:57
Your trying to get data from one dateframe to a different dataframe, which isn't the same as performing in dataframe calculations with a loop. — Trenton McKinney, Aug 25 '20 at 07:03
I was hoping there was a pretty solution to this, but it is always possible that there isn't. Thanks for your help. — Duco, Aug 25 '20 at 07:15
If it answers the question, please accept the answer. If an answer you like better comes along. you can unaccept an answer and accept a new answer. Have a nice day. Thank you. — Trenton McKinney, Aug 25 '20 at 07:30

Trenton McKinney · Answer 1 · 2020-08-24T21:10:22.027

Multi-column names are passed as Tuples, like df[('Foo', 'End')].

import pamadas as pd

# test data
col = pd.MultiIndex.from_arrays([['Foo', 'Bar'], ['Begin', 'Begin']], names=['First', 'Second'])
df = pd.DataFrame([[5, 1], [4, 4], [6, 6]], columns=col)
new = pd.DataFrame({'Foo': [1, 5, 7], 'Bar': [2, 4, 7]})

# write new columns
df[('Foo', 'End')] = new['Foo']
df[('Bar', 'End')] = new['Bar']

# display(df)
First    Foo   Bar Foo Bar
Second Begin Begin End End
0          5     1   1   2
1          4     4   5   4
2          6     6   7   7

For many columns

col, column name in new, must correspond to the top level column name in df.

for col in new.columns:
    df[(col, 'new col name')] = new[col]

Add multiple columns at once to multiindex Pandas dataframe

1 Answers1

For many columns