Denormalizing a DataFrame of company names [Part 1]

Question

I have a Pandas DataFrame of company names which has the following structure:

import numpy as np
import pandas as pd

df = pd.DataFrame({'name' : ['Nitron', 'Pulset', 'Rotaxi'], 
                   'postal_code' : [1410, 1020, 1310], 
                   'previous_name1' : ['Rotory', np.NaN, 'Datec'],
                   'previous_name2' : [ np.NaN, 'Cmotor', np.NaN],
                   'previous_name3' : ['Datec', np.NaN, np.NaN]
                  })

print(df)

| name   | postal_code | previous_name1 | previous_name2 | previous_name3 |
|--------|-------------|----------------|----------------|----------------|
| Nitron | 1410        | Rotory         | NaN            | Datec          |
| Pulset | 1020        | NaN            | Cmotor         | NaN            |
| Rotaxi | 1310        | Cyclip         | NaN            | NaN            |

As you'll notice, a company can have up to three previous names.

My goal is to "denormalize" the above table so that the new DataFrame has the following form:

| name   | postal_code |
|--------|-------------|
| Nitron | 1410        |
| Rotory | 1410        |
| Datec  | 1410        |
| Pulset | 1020        |
| Cmotor | 1020        |
| Rotaxi | 1310        |
| Cyclip | 1310        |

That is, I want to add a new row for all instances where the previous company names is non-missing and delete the previous names Series afterwards (I also want to add the postal_code value for each new row).

I'm looking for a description of the method (preferably with code or pseudocode) which will allow me to achieve the above result.

jezrael · Accepted Answer · 2019-07-07T14:09:26.983

Use DataFrame.set_index with DataFrame.stack for remove misisng values and reshape, then remove second level of MultiIndex by DataFrame.reset_index and last convert Series to 2 column DataFrame:

df1 = (df.set_index('postal_code')
         .stack()
         .reset_index(level=1, drop=True)
         .reset_index(name='name'))
print (df1)
   postal_code    name
0         1410  Nitron
1         1410  Rotory
2         1410   Datec
3         1020  Pulset
4         1020  Cmotor
5         1310  Rotaxi
6         1310   Datec

Or use DataFrame.melt with DataFrame.dropna, but ordering of values is different:

df1 = (df.melt('postal_code', value_name='name')
         .drop('variable', axis=1)
         .dropna(subset=['name'])
         .reset_index( drop=True)
)
print (df1)
   postal_code    name
0         1410  Nitron
1         1020  Pulset
2         1310  Rotaxi
3         1410  Rotory
4         1310   Datec
5         1020  Cmotor
6         1410   Datec

But possible sorting by first column:

df1 = (df.melt('postal_code', value_name='name')
         .drop('variable', axis=1)
         .dropna(subset=['name'])
         .sort_values('postal_code')
         .reset_index( drop=True)

)
print (df1)
   postal_code    name
0         1020  Pulset
1         1020  Cmotor
2         1310  Rotaxi
3         1310   Datec
4         1410  Nitron
5         1410  Rotory
6         1410   Datec

Denormalizing a DataFrame of company names [Part 1]

1 Answers1

Linked