6

Hi all I have a csv file which contains data as the format below

A   a
A   b
B   f
B   g
B   e
B   h
C   d
C   e
C   f

The first column contains items second column contains available feature from feature vector=[a,b,c,d,e,f,g,h] I want to convert this to occurence matrix look like below

    a,b,c,d,e,f,g,h
A   1,1,0,0,0,0,0,0
B   0,0,0,0,1,1,1,1
C   0,0,0,1,1,1,0,0

Can anyone tell me how to do this using pandas?

Isura Nirmal
  • 777
  • 1
  • 9
  • 26

3 Answers3

10

Here is another way to do it using pd.get_dummies().

import pandas as pd

# your data
# =======================
df

  col1 col2
0    A    a
1    A    b
2    B    f
3    B    g
4    B    e
5    B    h
6    C    d
7    C    e
8    C    f

# processing
# ===================================
pd.get_dummies(df.col2).groupby(df.col1).apply(max)

      a  b  d  e  f  g  h
col1                     
A     1  1  0  0  0  0  0
B     0  0  0  1  1  1  1
C     0  0  1  1  1  0  0
Jianxun Li
  • 24,004
  • 10
  • 58
  • 76
  • Shouldn't the apply function use a "sum" for completeness to record multiple values of a particular pair? It will also give the correct answer in the above case. – Arjun Mishra Oct 28 '20 at 00:58
8

Unclear if your data has a typo or not but you can crosstab for this:

In [95]:
pd.crosstab(index=df['A'], columns = df['a'])

Out[95]:
a  b  d  e  f  g  h
A                  
A  1  0  0  0  0  0
B  0  0  1  1  1  1
C  0  1  1  1  0  0

In your sample data your second column has value a as the name of that column but in your expected output it's in the column as a value

EDIT

OK I fixed your input data so it generates the correct result:

In [98]:
import pandas as pd
import io
t="""A   a
A   b
B   f
B   g
B   e
B   h
C   d
C   e
C   f"""
df = pd.read_csv(io.StringIO(t), sep='\s+', header=None, names=['A','a'])
df

Out[98]:
   A  a
0  A  a
1  A  b
2  B  f
3  B  g
4  B  e
5  B  h
6  C  d
7  C  e
8  C  f

In [99]:
ct = pd.crosstab(index=df['A'], columns = df['a'])
ct

Out[99]:
a  a  b  d  e  f  g  h
A                     
A  1  1  0  0  0  0  0
B  0  0  0  1  1  1  1
C  0  0  1  1  1  0  0
EdChum
  • 376,765
  • 198
  • 813
  • 562
0

This approach yields the same result in a scipy sparse coo matrix much faster

from scipy import sparse

df['col1'] = df['col1'].astype("category")
df['col2'] = df['col2'].astype("category")
df['ones'] = 1
user_items = sparse.coo_matrix((df.ones.astype(float),
                               (df.col1.cat.codes,
                                df.col2.cat.codes)))
Ofer Helman
  • 714
  • 1
  • 8
  • 25