Create flat ndarray from DataFrame column containing arrays

Question

I have a DataFrame with a column including numpy arrays:

df = pd.DataFrame({'Arrays': [np.array([1, 2, 3]), 
                           np.array([4, 5, 6]),
                           np.array([7, 8, 9])]})`

I need to extract the data in a way that I get a 2-dimensional array out of it. Like this:

array([[1, 2, 3],
    [4, 5, 6],
    [7, 8, 9]])`

But df.values results in an array containing a list of arrays:

array([[array([1, 2, 3])],
    [array([4, 5, 6])],
    [array([7, 8, 9])]], dtype=object)`

Is there an operator for this kind of problem or do I need to loop over all entries?

Are you sure it's a duplicate @coldspeed? The question you pointed to is about flattening into 1-dimensional array whereas the OP clearly mentions 2 dimensions. — ayorgo, Dec 13 '18 at 17:44

Sheldore · Accepted Answer · 2018-12-13T18:18:36.900

3

Use a combination of concatenate and flatten to get a single flattened array out of array of arrays

np.concatenate(df.values.flatten())

Alternative is to use ravel

np.concatenate(df.values.ravel())

Example

import pandas as pd
import numpy as np
df = pd.DataFrame({'Arrays': [np.array([1, 2, 3]), 
                               np.array([4, 5, 6]),
                               np.array([7, 8, 9])]})

np.concatenate(df.values.flatten())

# array([1, 2, 3, 4, 5, 6, 7, 8, 9])

Finally, to get a 2d array, you can simply use reshape as

np.reshape(flattened_array, (len(df),len(df)))

edited Dec 13 '18 at 18:18

answered Dec 13 '18 at 17:19

Sheldore

37,862
7
57
71

Thanks, that worked fine for me. – MetalPesto Dec 13 '18 at 18:15
@MetalPesto: You are welcome. Glad to help :) – Sheldore Dec 13 '18 at 18:15
@MetalPesto: If you untimately want a 2d array, you can now simple reshape it using `np.reshape` – Sheldore Dec 13 '18 at 18:16

ayorgo · Answer 2 · 2018-12-13T17:45:49.880

0

Apart from

df.applymap(list).values.ravel()

you can also do

df.applymap(list).values.reshape((-1,))

giving you

array([list([1, 2, 3]), list([4, 5, 6]), list([7, 8, 9])], dtype=object)

or, otherwise

df.applymap(list)['Arrays'].tolist()

resulting in

[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

edited Dec 13 '18 at 17:45

answered Dec 13 '18 at 17:27

ayorgo

2,803
2
25
35

Create flat ndarray from DataFrame column containing arrays

2 Answers2