One-hot encoding for list variable with customized delimiter and new column names

Question

My data:

Rank    Platforms        Technology

high    Windows||Linux   Unity
high    Linux             
low     Windows          Unreal 
low     Linux||MacOs     GameMakerStudio||Unity||Unreal
low                      GameMakerStudio
low

I want to convert it to something like this:

Rank    platform_Windows  platform_linux  platform_MacOs technology_unity  technology_unreal technology_GameMakerStudio

high    1                 0                0             1                  0                   1
high    0                 1                0             0                  0                   0
low     1                 0                0             0                  1                   0 
low     0                 1                1             1                  1                   1 
low     0                 0                0             0                  0                   1
low     0                 0                0             0                  0                   0

So it's sort of one-hot encoding. I have followed many answers:

The issues are:

none of them shows how to separate my list by || delimiter
none of them shows how to prefix the new column name. For example platform_ and technology_. I need this to know which original column the new column comes from.

My current code is:

df.drop('Platforms', 1).join(
    pd.get_dummies(
        pd.DataFrame(df.Platforms.str.split("||").tolist()).stack(),
        prefix=['platform']
    ).assum(level=0)
)

df.drop('Technology', 1).join(
    pd.get_dummies(
        pd.DataFrame(df.Technology.str.split("||").tolist()).stack(),
        prefix=['technology']
    ).assum(level=0)
)

But the error I get is:

TypeError: object of type 'float' has no len()

I have read the document pandas.get_dummies and pandas.Series.str.get_dummies. The latter seems to accept a customized delimiter while the former allows customized new column prefixes...

`df.Platforms.str.get_dummies()` seems to work as exptected. — Quang Hoang, Nov 19 '19 at 18:55
@Quang Hoang it does not allow prefix to column name `platform_` and `technology_` — hydradon, Nov 19 '19 at 18:56
Don't be lazy, just chain it: `df.Platforms.str.get_dummies().add_prefix('Platform_')` :D — Quang Hoang, Nov 19 '19 at 18:57
Oh, at a second look, it yields a `None` column. Which could be drop afterwards. — Quang Hoang, Nov 19 '19 at 18:58
@QuangHoang could you post a full working code as answer? Thank you — hydradon, Nov 19 '19 at 19:00
@DanielMesejo just similar example, here I mention 2 issues: column prefix and customize delimiter — hydradon, Nov 19 '19 at 19:10

score 2 · Accepted Answer · answered Nov 19 '19 at 19:03

2

You can do:

s = [df[col].str.get_dummies().add_prefix(f'{col.lower()}_') 
        for col in ['Platforms', 'Technology']]

pd.concat([df[['Rank']]] + s, axis=1)

answered Nov 19 '19 at 19:03

Quang Hoang

146,074
10
56
74

One-hot encoding for list variable with customized delimiter and new column names

1 Answers1