Pandas: list of lists to expanded rows

Question

I have an extension to this question. I have lists of lists in my columns and I need to expand the rows one step further. If I just repeat the steps it splits my strings into letters. Could you suggest a smart way around? Thanks!

d1 = pd.DataFrame({'column1': [['ana','bob',[1,2,3]],['dona','elf',[4,5,6]],['gear','hope',[7,8,9]]],
                   'column2':[10,20,30],
                  'column3':[44,55,66]})

d2 = pd.DataFrame.from_records(d1.column1.tolist()).stack().reset_index(level=1, drop=True).rename('column1')

d1_d2 = d1.drop('column1', axis=1).join(d2).reset_index(drop=True)[['column1','column2', 'column3']]

d1_d2

score 4 · Accepted Answer · answered Jan 28 '18 at 20:43

It seems you need flatten nested lists:

from collections import Iterable

def flatten(coll):
    for i in coll:
            if isinstance(i, Iterable) and not isinstance(i, str):
                for subc in flatten(i):
                    yield subc
            else:
                yield i

d1['column1'] = d1['column1'].apply(lambda x: list(flatten(x)))
print (d1)
                 column1  column2  column3
0    [ana, bob, 1, 2, 3]       10       44
1   [dona, elf, 4, 5, 6]       20       55
2  [gear, hope, 7, 8, 9]       30       66

And then use your solution:

d2 = (pd.DataFrame(d1.column1.tolist())
        .stack()
        .reset_index(level=1, drop=True)
        .rename('column1'))

d1_d2 = (d1.drop('column1', axis=1)
          .join(d2)
          .reset_index(drop=True)[['column1','column2', 'column3']])

print (d1_d2)
   column1  column2  column3
0      ana       10       44
1      bob       10       44
2        1       10       44
3        2       10       44
4        3       10       44
5     dona       20       55
6      elf       20       55
7        4       20       55
8        5       20       55
9        6       20       55
10    gear       30       66
11    hope       30       66
12       7       30       66
13       8       30       66
14       9       30       66

score 0 · Answer 2 · answered Sep 20 '20 at 15:20

Assuming the expected result is same as jezrael.

pandas >= 0.25.0

d1 = d1.explode('column1').explode('column1').reset_index(drop=True)

d1:

   column1  column2  column3
0      ana       10       44
1      bob       10       44
2        1       10       44
3        2       10       44
4        3       10       44
5     dona       20       55
6      elf       20       55
7        4       20       55
8        5       20       55
9        6       20       55
10    gear       30       66
11    hope       30       66
12       7       30       66
13       8       30       66
14       9       30       66

Pandas: list of lists to expanded rows

2 Answers2

pandas >= 0.25.0