Adding binary flag to pandas DataFrame

Question

This is the continuation of my previous post on denormalizing a DataFrame of company names.

SOME BACKGROUND:

Initially, I was working with the table below:

import numpy as np
import pandas as pd

df = pd.DataFrame({'name' : ['Nitron', 'Pulset', 'Rotaxi'], 
                   'postal_code' : [1410, 1020, 1310], 
                   'previous_name1' : ['Rotory', np.NaN, 'Datec'],
                   'previous_name2' : [ np.NaN, 'Cmotor', np.NaN],
                   'previous_name3' : ['Datec', np.NaN, np.NaN],
                   'country' : ['BEL', 'ENG', 'JPN'], 
                   'city' : ['Brussels', np.NaN, np.NaN]
                  })

print(df)

| name   | postal_code | previous_name1 | previous_name2 | previous_name3 | country | city     |
|--------|-------------|----------------|----------------|----------------|---------|----------|
| Nitron | 1410        | Rotory         | NaN            | Datec          | BEL     | Brussels |
| Pulset | 1020        | NaN            | Cmotor         | NaN            | ENG     | NaN      |
| Rotaxi | 1310        | Cyclip         | NaN            | NaN            | JPN     | NaN      |

The goal of the denormalization was to add a new row for all instances where the previous company name was non-missing and delete all previous name Series afterwards.

Thanks to jezrael, I was able to achieve this with the following code:

df1 = (df.set_index(['postal_code','country','city'])
         .stack()
         .reset_index(level=3, drop=True)
         .reset_index(name='name')
         )

print (df1)

   postal_code country      city    name
0         1410     BEL  Brussels  Nitron
1         1410     BEL  Brussels  Rotory
2         1410     BEL  Brussels   Datec
3         1020     ENG       NaN  Pulset
4         1020     ENG       NaN  Cmotor
5         1310     JPN       NaN  Rotaxi
6         1310     JPN       NaN   Datec

NEW GOAL

My new goal is to add an additional flag/column with values defined as

1 if the company name is an old one (i.e. from one of the previous name Series);
0 otherwise.

That is, the new DataFrame should visually look like this (the order of the columns do not matter):

| name   | postal_code | country | city     | old_name_flag |
|--------|:-----------:|:-------:|----------|:-------------:|
| Nitron |     1410    |   BEL   | Brussels |       0       |
| Rotory |     1410    |   BEL   | Brussels |       1       |
| Datec  |     1410    |   BEL   | Brussels |       1       |
| Pulset |     1020    |   ENG   | NaN      |       0       |
| Cmotor |     1020    |   ENG   | NaN      |       1       |
| Rotaxi |     1310    |   JPN   | NaN      |       0       |
| Cyclip |     1310    |   JPN   | NaN      |       1       |

I tried to adjust the code of jezrael without success. Any additional help would be greatly appreciated.

BENY · Answer 1 · 2019-07-09T19:56:47.360

Here is one way using wide_to_long

df=df.rename({'name':'previous_name0'})
s=pd.wide_to_long(df,['previous'],i=['postal_code','country','city'],j='old_name_flag',sep='_',suffix='\w+').reset_index()
s=s[s.previous.notnull()]
s['old_name_flag']=s['old_name_flag'].ne('name0').astype(int)
s
Out[147]: 
   postal_code country      city  old_name_flag previous
0         1410     BEL  Brussels              0   Nitron
1         1410     BEL  Brussels              1   Rotory
3         1410     BEL  Brussels              1    Datec
4         1020     ENG       NaN              0   Pulset
6         1020     ENG       NaN              1   Cmotor
8         1310     JPN       NaN              0   Rotaxi
9         1310     JPN       NaN              1    Datec

Or we start from your df1

df1['old_name_flag']=df.groupby(['postal_code','country']).cumcount().ne(0).astype(int)
df1
Out[152]: 
   postal_code country      city    name  old_name_flag
0         1410     BEL  Brussels  Nitron              0
1         1410     BEL  Brussels  Rotory              1
2         1410     BEL  Brussels   Datec              1
3         1020     ENG       NaN  Pulset              0
4         1020     ENG       NaN  Cmotor              1
5         1310     JPN       NaN  Rotaxi              0
6         1310     JPN       NaN   Datec              1

Quang Hoang · Accepted Answer · 2019-07-09T20:06:48.397

The info is already in the dataframe when you reindex:

df1 = (df.set_index(['postal_code','country','city'])
         .stack()
#          .reset_index(level=3, drop=True)
         .reset_index(name='name')
         )

df1['old_name_flag'] = df1['level_3'].ne('name')
df1.drop('level_3', axis=1, inplace=True)

print (df1)

Output:

   postal_code country      city    name  old_name_flag
0         1410     BEL  Brussels  Nitron          False
1         1410     BEL  Brussels  Rotory           True
2         1410     BEL  Brussels   Datec           True
3         1020     ENG       NaN  Pulset          False
4         1020     ENG       NaN  Cmotor           True
5         1310     JPN       NaN  Rotaxi          False
6         1310     JPN       NaN   Datec           True

Update: You can also chain command with assign as follows (though I prefer the separate command for clarity):

df1 = (df.set_index(['postal_code','country','city'])
         .stack()
         .reset_index(name='name')
         .assign(old_name_flag=lambda x: x['level_3'].ne('name'))
         .drop('level_3', axis=1)
         )

Adding binary flag to pandas DataFrame

2 Answers2

Linked