Pandas, DataFrame: Splitting one column into multiple columns

Question

I have the following DataFrame. I am wondering whether it is possible to break the data column into multiple columns. E.g., from this:

ID       Date       data
6       21/05/2016  A: 7, B: 8, C: 5, D: 5, A: 8
6       21/01/2014  B: 5, C: 5, D: 7
6       02/04/2013  A: 4, D:7
7       05/06/2014  C: 25
7       12/08/2014  D: 20
8       18/04/2012  A: 2, B: 3, C: 3, E: 5, B: 4
8       21/03/2012  F: 6, B: 4, F: 5, D: 6, B: 4

into this:

ID       Date       data                            A   B   C   D   E   F
6       21/05/2016  A: 7, B: 8, C: 5, D: 5, A: 8    15  8   5   5   0   0
6       21/01/2014  B: 5, C: 5, D: 7                0   5   5   7   0   0     
6       02/04/2013  B: 4, D: 7, B: 6                0   10  0   7   0   0
7       05/06/2014  C: 25                           0   0   25  0   0   0
7       12/08/2014  D: 20                           0   0   0   20  0   0   
8       18/04/2012  A: 2, B: 3, C: 3, E: 5, B: 4    2   7   3   0   5   0
8       21/03/2012  F: 6, B: 4, F: 5, D: 6, B: 4    0   8   0   6   0   11

I have tried this Split strings in tuples into columns, in Pandas, and this pandas: How do I split text in a column into multiple rows? but they are not working in my case.

EDIT

There is a bit of complexity the data column has duplicate values for example in first row A is repeated, and therefore these values are summed up under the A column (please see second table).

Psidom · Accepted Answer · 2016-07-14T21:41:12.673

Here is a function that can convert the string to a dictionary and aggregate values based on the key; After the conversion it will be easy to get the results with the pd.Series method:

def str_to_dict(str1):
    import re
    from collections import defaultdict
    d = defaultdict(int)
    for k, v in zip(re.findall('[A-Z]', str1), re.findall('\d+', str1)):
        d[k] += int(v)
    return d

pd.concat([df, df['dictionary'].apply(str_to_dict).apply(pd.Series).fillna(0).astype(int)], axis=1)

piRSquared · Answer 2 · 2016-07-14T21:43:55.410

5

df = pd.DataFrame([
        [6, "a: 1, b: 2"],
        [6, "a: 1, b: 2"],
        [6, "a: 1, b: 2"],
        [6, "a: 1, b: 2"],
    ], columns=['ID', 'dictionary'])

def str2dict(s):
    split = s.strip().split(',')
    d = {}
    for pair in split:
        k, v = [_.strip() for _ in pair.split(':')]
        d[k] = v
    return d

df.dictionary.apply(str2dict).apply(pd.Series)

Or:

pd.concat([df, df.dictionary.apply(str2dict).apply(pd.Series)], axis=1)

edited Jul 14 '16 at 21:43

answered Jul 14 '16 at 21:01

piRSquared

285,575
57
475
624

This will only give you a series and do not split into multiple columns. – user1124825 Jul 14 '16 at 21:23
@user1124825 I edited the answer to include a string parser. Your original question mentioned the column labeled `'dictionary'` was a column of dictionaries. I assumed that was true. By applying a parser, my same answer still holds. – piRSquared Jul 14 '16 at 21:45

Pandas, DataFrame: Splitting one column into multiple columns

2 Answers2

Linked