Pandas Pivot Table formatting column names

Question

I used the pandas.pivot_table function on a pandas dataframe and my output looks like something simillar to this:

                    Winners                 Runnerup            
         year       2016    2015    2014    2016    2015    2014
Country  Sport                              
india    badminton                              
india    wrestling  

What I actually needed was some thing like below

Country Sport   Winners_2016    Winners_2015    Winners_2014    Runnerup_2016   Runnerup_2015   Runnerup_2014
india   badminton   1   1   1   1   1   1
india   wrestling   1   0   1   0   1   0

I have lot of columns and years so I will not be able to manually edit them, so can anyone please advise me on how to do this ?

Answer 1

You can also use list comprehension:

df.columns = ['_'.join(col) for col in df.columns]
print (df)
                   Winners_2016  Winners_2015  Winners_2014  Runnerup_2016  \
Country Sport                                                                
india   badminton             1             1             1              1   
        wrestling             1             1             1              1   

                   Runnerup_2015  Runnerup_2014  
Country Sport                                    
india   badminton              1              1  
        wrestling              1              1  

Another solution with convert columns to_series and then call join:

df.columns = df.columns.to_series().str.join('_')
print (df)
                   Winners_2016  Winners_2015  Winners_2014  Runnerup_2016  \
Country Sport                                                                
india   badminton             1             1             1              1   
        wrestling             1             1             1              1   

                   Runnerup_2015  Runnerup_2014  
Country Sport                                    
india   badminton              1              1  
        wrestling              1              1  

I was really interested about timings:

In [45]: %timeit ['_'.join(col) for col in df.columns]
The slowest run took 7.82 times longer than the fastest. This could mean that an intermediate result is being cached.
100000 loops, best of 3: 4.05 µs per loop

In [44]: %timeit ['{}_{}'.format(x,y) for x,y in zip(df.columns.get_level_values(0),df.columns.get_level_values(1))]
The slowest run took 4.56 times longer than the fastest. This could mean that an intermediate result is being cached.
10000 loops, best of 3: 131 µs per loop

In [46]: %timeit df.columns.to_series().str.join('_')
The slowest run took 4.31 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 452 µs per loop

Answer 2

Try this:

df.columns=['{}_{}'.format(x,y) for x,y in zip(df.columns.get_level_values(0),df.columns.get_level_values(1))]

get_level_values is what you need to get only one of the levels of the resulting multiindex.

Side note: you might try working with the data as is. I really hated pandas multiIndex for a long time, but it's grown on me.