1

I am trying to use numpy.append but something goes wrong and it just doesn't make sence to me anymore. Can someone explain why I am getting an error?

>>> np.array([[], [], []]).shape
(3, 0)

>>> a=[[], [], []]
>>> a[1].append(3)
>>> a
[[], [3], []]

>>> b=np.array(a)
>>> b[0].append(3)
array([[3], [3], []], dtype=object)

This is all logical to me, yet when I try the following it stops working.

>>> c=np.array((3,0),dtype=object)
>>> c[0].append(3)
AttributeError: 'int' object has no attribute 'append'

????
>>> np.empty((3,1))[0].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'

>>> np.empty((3,0))[1].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'

>>>np.empty((6,1),dtype=object)[0].append(3)
AttributeError: 'numpy.ndarray' object has no attribute 'append'

Solved: How to create a numpy array of lists?

Community
  • 1
  • 1
Perm. Questiin
  • 429
  • 2
  • 9
  • 21

3 Answers3

3

Don't just look at the shape; check the dtype, and if object, the nature of the elements

In [282]: np.array([[], [], []])
Out[282]: array([], shape=(3, 0), dtype=float64)

A 2d array of floats. np.array tries to make a multidimensional array of numbers; it's only when it can't do that it makes an object array.

In [283]: b=np.array([[],[3],[]])
In [284]: b
Out[284]: array([[], [3], []], dtype=object)

Here the 3 sublists have different size, so it can't make a 2d array; the result is an object array, where the objects are lists, and have the append method.

In [286]: c=np.array((3,0), object)
In [287]: c
Out[287]: array([3, 0], dtype=object)

This is a (2,) object array; the 2 elements are numbers. Numbers don't have an append method.

In [288]: np.empty((3,1))
Out[288]: 
array([[ 0.],
       [ 0.],
       [ 0.]])

A (3,1) array of floats. No append method for numbers or arrays.

In [289]: np.empty((3,0))
Out[289]: array([], shape=(3, 0), dtype=float64)

Another 2d array of floats

In [290]: np.empty((6,1),object)
Out[290]: 
array([[None],
       [None],
       [None],
       [None],
       [None],
       [None]], dtype=object)

2d array of dtype object. In this case they are initialized to None. Again no append method.

More on making an array of lists

dimensions of array of arrays in numpy

and

How to keep numpy from broadcasting when creating an object array of different shaped arrays

In [305]: d=np.empty((3,),object)
In [306]: d
Out[306]: array([None, None, None], dtype=object)
In [307]: d.fill([])
In [308]: d
Out[308]: array([[], [], []], dtype=object)   # array of lists
In [309]: d[0].append([1,2,3])
In [310]: d
Out[310]: array([[[1, 2, 3]], [[1, 2, 3]], [[1, 2, 3]]], dtype=object)

But oops - those lists are all the same object (pointer) :( I have to put a different list in each element. Now I can append to them individually.

In [311]: d[...]=[[],[1,2,3],[2]]
In [312]: d
Out[312]: array([[], [1, 2, 3], [2]], dtype=object)
In [313]: d[0].append([2,3])
In [314]: d
Out[314]: array([[[2, 3]], [1, 2, 3], [2]], dtype=object)

I think you have to bite the bullet and use a list to initialize an object array of lists. There isn't a short cut:

In [319]: d=np.empty((3,),object)
In [320]: d[...]=[[] for _ in range(3)]
In [321]: d
Out[321]: array([[], [], []], dtype=object)
In [323]: d
Out[323]: array([[], [3], []], dtype=object)
Community
  • 1
  • 1
hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • Thank you for your responce. Do you have any idea why my last edit appends to all lists? – Perm. Questiin Apr 19 '17 at 00:21
  • For the same reason that `alist=[[]]*3; alist[0].append[1)` does the same thing. The elements are all the same list object. An object array, like a list, contains pointers to objects else where in memory. – hpaulj Apr 19 '17 at 00:40
3

I know this is a rather unusual use case but in fact, it can be very handy to have an (nd)array holding lists in each cell. Imho numpy should allow passing a lambda function to the fill method. But to achieve what you want, this is what I do:

m = np.empty((12, 12), dtype=object)
for i in np.ndindex(m.shape): m[i] = []
KIC
  • 5,887
  • 7
  • 58
  • 98
2

First of all, stop updating your question when someone answers it correctly. Accept/upvote an answer and ask a new question when you are ready. SO is a Q&A site that is meant to help future visitors viewing your question, and not just you. Invalidating all the good answers by changing the context completely for your own benefit defeats the purpose of this site, to say the least.

Secondly, np.array([[], [3], []]) ends up with dtype=object because it is a ragged array. [[], [], []] and [[3], [3], [3]], having a uniform length in all elements across all dimensions, would produce numerical arrays.

np.zeros((6,),dtype=object) produces an empty array of object references (containing NULLs). When you fill it with [], you are filling it with a reference to the same python list in every element. numpy has no knowledge of what object you are passing to ndarray.fill, so it does not call the constructor on the list type for each element as you seem to be expecting. It just copies the reference that you passed in six times. After that, it should be clear why changing the contents of that one list makes it appear that all the array elements have changed.

Mad Physicist
  • 107,652
  • 25
  • 181
  • 264
  • yes `fill` should accept a lamda function and eventually a parameter to stop bradcasting – KIC Jun 11 '20 at 06:26
  • @KIC. If you pass in a function (a lambda is just a simplified callable), it will copy the reference to the function. You need a different method from `fill` entirely – Mad Physicist Jun 11 '20 at 06:51
  • yes with should I mean it doesn't. but it should in the sense of a new feature. – KIC Jun 12 '20 at 07:21
  • @KIC. I'm not sure that I agree with that. If you want to do that, you can use `np.empty` and apply. – Mad Physicist Jun 12 '20 at 10:11