2
import pandas as pd

raw_data = {'FirstName': ["John", "Jill", "Jack", "John", "Jill", "Jack",],
            'LastName': ["Blue", "Green", "Yellow","Blue", "Green", "Yellow"],
            'Building': ["Building1", "Building1", "Building2","Building1", "Building1", "Building2"],
            'Month': ["November", "November", "November", "December","December", "December"], 
            'Sales': [100, 150, 275, 200, 150, 150]}

frame = pd.DataFrame(raw_data, columns =raw_data.keys())

df = frame.pivot(
    index=["FirstName", "LastName", "Building"],
    columns="Month",
    values="Sales",
)

df

I have some simple code here that generates a multi-level index dataframe that looks like this:

DataFrame1

I'm trying to adjust the column headers (specifically December and November) and move them down, so they are not so far above the dataframe with all that whitespace underneath them. Ideally I'd also like to hide the "Month" that shows up above "building" but one problem at a time.

One solution I've tried is just hiding the index like so.

df.index.names = (None, None, None)

df

DataFrame2

This datarame nicely moves the column headers down, so there's not as much whitespace. But now I've lost my "FirstName" and "LastName" headers since I was forced to set them to None. is there a different way of accomplishing? Can I directly control the whitespace that shows up beneath the "December" and the "November" in my first example?

Is such level of minute control possible with pandas?

Edit:

It seems like I might need to nuke my indexing by resetting it. In light of that I was wondering if there's a way to merge rows together in Pandas to preserve the visual element of indexing?

So for instance, the original data might look like this:

UnmergedNames

But, I was wondering if it's possible to merge, just visually rows in 1 column, like so:

MergedRows

So you can see here, that instead of "Jack" showing up in 2 rows, now there's just 1 merged cell. Indexing does something like this automatically, but if I remove indexing I'd like to mimic this feature, is that possible?

  • hi, perhaps could be done with https://pandas.pydata.org/docs/reference/api/pandas.io.formats.style.Styler.set_sticky.html – jspcal Jan 12 '22 at 18:50
  • 1
    See my answer here for how pandas organizes index and column labels with named axes: https://stackoverflow.com/questions/55027108/pandas-rename-index. The issue is that with names for your index levels, the column labels are placed one row above in the display, that way it's possible to distinguish between a level of a named Index and a column in your DataFrame, as these are inherently 2 very different objects – ALollz Jan 12 '22 at 18:50
  • `df = df.reset_index()`. – Quang Hoang Jan 12 '22 at 18:52
  • @ALollz I see. That makes sense. I can collapse it all by just resetting the index, but I quite like the way it looks in my actual data, because it stacks people with the same together, which is really nice, similar to how you might merge rows in excel. Ok, I might just have to do it in excel. Thank you very much! – Pewter City Jan 12 '22 at 18:58
  • @QuangHoang hi, I had looked into collapsing the index alltogether, but I was trying to avoid it if it that was possible. I might just have to go down this path though. – Pewter City Jan 12 '22 at 18:58
  • @PewterCity Yeah, I'd take QuangHoang's solution to reset_index if you aren't really using the `index` for anything. On the other hand, the Index is used for alignment, so if you are using it to align operations then `.reset_index` will force you to deal with merges, etc. – ALollz Jan 12 '22 at 18:59
  • @ALollz yeah, I think that might be the way to go. It's a shame, my actual data consists of hundreds of rows with many columns, so instead of seeing "Jack" 20 consecutive times, each with it's own row, all 20 of those rows were just merged as one, with "Jack" that was centered so i was hoping to try and preserve that indexing. At any rate, thank you very much for your time! – Pewter City Jan 12 '22 at 19:01

1 Answers1

1

Try:

df = frame.pivot(
    index=["FirstName", "LastName", "Building"],
    columns="Month",
    values="Sales",
).rename_axis(columns=None).reset_index()

Output:

>>> df
  FirstName LastName   Building  December  November
0      Jack   Yellow  Building2       150       275
1      Jill    Green  Building1       150       150
2      John     Blue  Building1       200       100

Update

Suppose the following dataframe:

>>> df
  FirstName LastName   Building  December  November
0      Jack   Yellow  Building2       150       275
1      Jack   Yellow  Building1       125       175
2      John     Blue   Building       200       100

You can do something like this:

df.loc[df['FirstName'].eq(df['FirstName'].shift()), 'FirstName'] = ''
print(df)

# Output
  FirstName LastName   Building  December  November
0      Jack   Yellow  Building2       150       275
1             Yellow  Building1       125       175
2      John     Blue   Building       200       100

I checked if the first name of current row is the same than the previous one. If yes, I replace the first name by ''. So when (index 1, Jack) == (index 0, Jack), I set (index 1, Jack) to ''

Corralien
  • 109,409
  • 8
  • 28
  • 52
  • Hi, I think you might be right that indexing rows is my only play. In light of that, do you know if there's any way to merge rows together? I editted my question with some pictures to make it a little clearer what I'm asking. The nice thing about indexing was visually it "clumped" certain rows together, which made it nicer to look at. I was hoping to mimic that somehow, but without creating the indexing and thus creating the height problem of the collumn headers all over again. – Pewter City Jan 12 '22 at 19:20
  • Hi. Thank you very much! I think this will solve my issue. Could you possibly explain what you did? I don't totally follow. So first, you reset the index to get the first dataframe. But what did you do with the .eq method? Did you make it so that if there's a duplicate in a column, it deletes it? – Pewter City Jan 12 '22 at 19:36
  • I updated my answer. Are my explanation are clear? (English is not my best skill :)) – Corralien Jan 12 '22 at 19:40
  • Yup it makes sense, thank you very much! – Pewter City Jan 12 '22 at 19:44