I'm unable to convert a list to an numpy array

Question

I need to convert 2 list to 2 numpy array, it work with one of the array but not the other. I get the error message 'ValueError: could not broadcast input array from shape (2) into shape (1)'

list1 =[array([[-0.01488726],
       [ 0.22409286]]), array([[0.0618533]]), array([[1.20952571e-01, 1.32928502e-01, 1.06338076e-01, 1.16971096e-01,
        2.61011127e+05, 1.14151376e-01, 1.14312841e-01, 1.05826116e-01,
        1.07342306e-01, 6.35733298e+04, 1.09819405e-01, 1.16971096e-01,
        1.06503478e-01, 1.16971096e-01, 2.74499917e+04, 1.12977806e-01,
        1.15962928e-01, 1.06338076e-01, 1.06338076e-01, 2.49367238e+04,
        1.06657066e-01, 1.11485245e-01, 1.06338076e-01, 1.09153857e-01,
        2.02111931e+04, 1.08669464e-01, 1.10315613e-01, 1.08334721e-01,
        1.09508291e-01, 7.21720168e+03, 1.08992393e-01, 1.12828156e-01,
        1.07826699e-01, 1.11969638e-01, 1.70042704e+04, 1.11646710e-01,
        1.14151376e-01, 1.08826990e-01, 1.12158670e-01, 1.79861313e+04,
        1.12158670e-01, 1.15482473e-01, 1.11343472e-01, 1.14151376e-01,
        1.23411792e+04, 1.16301609e-01, 1.19621475e-01, 1.14816925e-01,
        1.17301901e-01, 1.28108082e+04, 1.19467886e-01, 1.21460593e-01,
        1.17786294e-01, 1.18298254e-01, 1.26929667e+04, 1.20298837e-01,
        1.23453300e-01, 1.20133435e-01, 1.21622058e-01, 9.39268212e+03,
        1.21622058e-01, 1.22287606e-01, 1.17786294e-01, 1.20298837e-01,
        9.24742325e+03, 1.20298837e-01, 1.21275500e-01, 1.19125267e-01,
        1.20133435e-01, 7.87820533e+03, 1.20298837e-01, 1.22130080e-01,
        1.20298837e-01, 1.21783522e-01, 8.46741247e+03, 1.21460593e-01,
        1.22130080e-01, 1.18967741e-01, 1.18967741e-01, 6.94707571e+03,
        1.20952571e-01, 1.22953154e-01, 1.17979264e-01, 1.20952571e-01,
        1.67779170e+04, 1.20645395e-01, 1.20645395e-01, 1.11808174e-01,
        1.14312841e-01, 9.71880078e+03, 1.11646710e-01, 1.12477660e-01,
        1.08992393e-01, 1.10827573e-01, 1.21760681e+04, 1.11646710e-01,
        1.25611409e-01, 1.11646710e-01, 1.25611409e-01, 1.07829180e+04,
        1.23295774e-01, 1.25264851e-01, 1.20798983e-01, 1.22130080e-01,
        1.54740173e+04, 1.22953154e-01, 1.24788335e-01, 1.22610535e-01,
        1.24626870e-01, 4.99446731e+03, 1.24118848e-01, 1.24118848e-01,
        1.22287606e-01, 1.23933755e-01, 9.42885802e+03, 1.23933755e-01,
        1.26430545e-01, 1.23110680e-01, 1.26430545e-01, 7.95286489e+03,
        1.26276957e-01, 1.32928502e-01, 1.26119431e-01, 1.32251139e-01,
        9.01172070e+03, 1.32763100e-01, 1.41403413e-01, 1.31750994e-01,
        1.40387369e-01, 8.46975961e+03, 1.44215256e-01, 1.63488589e-01,
        1.43396120e-01, 1.63488589e-01, 1.48561956e+04, 1.67481879e-01,
        2.10175423e-01, 1.67320414e-01, 1.91402236e-01, 4.38947840e+04,
        2.02035256e-01, 2.12672214e-01, 1.97388232e-01, 2.10679507e-01,
        2.78391169e+04, 2.11345055e-01, 2.12526502e-01, 2.02708680e-01,
        2.05524461e-01, 8.05903756e+03]])]
list2 = [array([[-0.03220841,  0.48482216]]), array([[0.0618533]]), array([[0.00393816]])]

arr1 = np.array(list1)#this work
arr2 = np.array(list2)#this doesn't work

in list2 = [array([[-0.03220841, 0.48482216]]), does array mean np.array? you are referring to? — Surya Tej, May 20 '20 at 22:59
Have you referred this thread ? https://stackoverflow.com/questions/43977463/valueerror-could-not-broadcast-input-array-from-shape-224-224-3-into-shape-2 — Surya Tej, May 20 '20 at 23:07
Yes but I fon't see any item that aren't in the right dimension. — M D, May 20 '20 at 23:16

Burnished Sandalwood · Answer 1 · 2020-05-20T23:29:55.977

If you want all entries from the list of arrays to be in a single array you can you list comprehension to generate a np array of list2 and then use .ravel to flatten it into a single array.

myarray = [np.concatenate(i) for i in list2]
myarray = np.concatenate(myarray).ravel()
print(myarray)

>>>[-0.03220841  0.48482216  0.0618533   0.00393816]

If you instead want an array of arrays, you can use the .asarray() command

myarray = [np.concatenate(i) for i in list2]
myarray = np.asarray(myarray)
print(myarray)

>>>array([array([-0.03220841,  0.48482216]), array([0.0618533]),
       array([0.00393816])], dtype=object)

hpaulj · Answer 2 · 2020-05-20T23:39:47.547

Your lists contain arrays of various sizes. If np.array can't make a multidimensional array, it has 2 fall back options. One is to make an object dtype array. But with some combinations of dimensions (1st all the same) it throws an error.

In [495]: [i.shape for i in list1]                                                       
Out[495]: [(2, 1), (1, 1), (1, 150)]
In [496]: [i.shape for i in list2]                                                       
Out[496]: [(1, 2), (1, 1), (1, 1)]

Make an array from list1 - note the shape. dtype is `object:

In [497]: np.array(list1).shape                                                          
Out[497]: (3,)

But with list2

In [498]: np.array(list2).shape                                                          
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-498-0846273caa0f> in <module>
----> 1 np.array(list2).shape

ValueError: could not broadcast input array from shape (2) into shape (1)

This is because the component arrays all have shape (1,n).

np.array(alist) is not a reliable operation when the list can contain lists or arrays of unknown dimensions. While there are ways consistently making an object dtype array, I question the wisdom of doing so.

Why do you want to make an array from these lists? Why not leave as lists?

I need to make matrix multiplication and others operation with it so leaving them as list would slow down dramatically my code. — M D, May 21 '20 at 01:12
But you can't do matrix multiplication with "irregular" arrays like this. You need a 2d array. Making an array from `list1` does not help you. — hpaulj, May 21 '20 at 03:15

I'm unable to convert a list to an numpy array

2 Answers2