1

I am trying to generate a numpy array with elements as two other numpy arrays, as below.

W1b1 = np.zeros((256, 161))
W2b2 = np.zeros((256, 257))
Wx = np.array([W1b1, W2b2], dtype=np.object)

this gives an error:

ValueError: could not broadcast input array from shape (256,161) into shape (256).

However, if I take entirely different dimensions for of W1b1 and W2b2 then I do not get an error, as below.

A1 = np.zeros((256, 161))
A2 = np.zeros((257, 257))
A3 = np.array([A1, A2], dtype=np.object)

I do not get what is wrong in the first code and why is numpy array trying to broadcast one of the input arrays.

I have tried on below versions (Python 2.7.6, Numpy 1.13.1) and (Python 3.6.4, Numpy 1.14.1).

Nico Schlömer
  • 53,797
  • 27
  • 201
  • 249
Gaurav Srivastava
  • 505
  • 1
  • 7
  • 17
  • Why not just `Wx = [W1b1, W2b2]`? There's no benefit to keeping dtype object arrays anyway. – cs95 Jun 23 '18 at 23:11
  • 1
    If you _really_ want to do this, an empty array will stop NumPy from trying to broadcast on the first dimension. `np.array([W1b1, W2b2, np.array([])], dtype=object)` – miradulo Jun 23 '18 at 23:20
  • 1
    It is documented as a bug of sorts here: [Inconsistent behaviour of numpy.asarray on list of arrays #7453](https://github.com/numpy/numpy/issues/7453) – miradulo Jun 23 '18 at 23:27

1 Answers1

2

Don't count on np.array(..., object) making the right object array. At the moment we don't have control over how many dimensions it makes. Conceivably it could make a (2,) array, or (2, 256) (with 1d contents). Sometimes it works, sometimes raises an error. There's something of a pattern, but I haven't seen an analysis of the code that shows exactly what is happening.

For now it is safer to allocate the array, and fill it:

In [57]: arr = np.empty(2, object)
In [58]: arr[:] = [W1b1, W2b2]

np.array([np.zeros((3,2)),np.ones((3,4))], object) also raises this error. So the error arises when the first dimensions match, but the later ones don't. Now that I think about, I've seen this error before.

Earlier SO questions on the topic

numpy array 1.9.2 getting ValueError: could not broadcast input array from shape (4,2) into shape (4)

Creation of array of arrays fails, when first size of first dimension matches

Creating array of arrays in numpy with different dimensions

hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • It is suggested by user miradulo that this is possibly a bug. If you could elaborate if this a bug or expected feature of numpy. This will make the answer more complete. – Gaurav Srivastava Jun 24 '18 at 01:09
  • This an edge case whose behavior changed around version 1.9, and may change in the future. I haven't followed the relevant `issues` discussion. Keep in mind that the basic `numpy` array is multidimensional with numeric or structured `dtype`. Object dtype is a generalization that makes the array more list-like. The desired behavior for cases like yours is not well defined. – hpaulj Jun 24 '18 at 04:10