Tracking Multi-Index and Modifying Values in NumPy

Question

I have a 2D array that I am iterating over in an effort to use the index values to make calculations and then assign the calculated value to said index.

In the NumPy documentation, an example is provided for modifying values using an iterator:

for x in np.nditer(a, op_flags=['readwrite']):
    x[...] = 2 * x

However, this doesn't seem to work when tracking an index using the following method:

it = np.nditer(a, flags=['multi_index'])
while not it.finished:
    it[...] = . . .
    it.iternext()

I am, however, able to use the it.multi_index values, but it seems unnecessarily verbose. Is there a simpler way to achieve this, either through a different approach or different syntax?

it = np.nditer(a, flags=['multi_index'])
while not it.finished:
    matrix[it.multi_index[0]][it.multi_index[1]] = . . .
    it.iternext()

EDIT

Here is an example of a multi_index iteration attempting to modify values using iterator indexing and failing.

matrix = np.zeros((5,5))
it = np.nditer(matrix, flags=['multi_index'])
while not it.finished:
  it[...] = 1
  it.iternext()

The error produced is

TypeError                                 Traceback (most recent call last)
<ipython-input-79-3f4cabcbfde6> in <module>()
     25 it = np.nditer(matrix, flags=['multi_index'])
     26 while not it.finished:
---> 27   it[...] = 1
     28   it.iternext()

TypeError: invalid index type for iterator indexing

"this doesn't seem to work" this kind of statements are better when followed by a [mcve]... — Julien, Feb 12 '18 at 00:14

hpaulj · Accepted Answer · 2018-02-12T02:15:25.607

In your first iteration example:

In [1]: arr = np.arange(12).reshape(3,4)
In [2]: for x in np.nditer(arr, op_flags=['readwrite']):
   ...:     print(x, type(x))
   ...:     x[...] = 2 * x
   ...:     
0 <class 'numpy.ndarray'>
1 <class 'numpy.ndarray'>
2 <class 'numpy.ndarray'>
3 <class 'numpy.ndarray'>
4 <class 'numpy.ndarray'>
....
11 <class 'numpy.ndarray'>
In [3]: x
Out[3]: array(22)
In [4]: arr
Out[4]: 
array([[ 0,  2,  4,  6],
       [ 8, 10, 12, 14],
       [16, 18, 20, 22]])

Turn on multi_index:

In [9]: it = np.nditer(arr, flags=['multi_index'],op_flags=['readwrite'])
In [10]: while not it.finished:
    ...:     print(it[0], it.multi_index)
    ...:     it.iternext()
    ...:     
0 (0, 0)
2 (0, 1)
4 (0, 2)
...
20 (2, 2)
22 (2, 3)

Same iteration through the elements of arr, but is also generates the 2d index tuple. it is the nditer object, with various methods and attributes. In this case it has a multi_index attribute. And the current iteration variable is in it[0].

I can modify elements either with the [...] inplace, or by indexing in arr:

In [11]: it = np.nditer(arr, flags=['multi_index'],op_flags=['readwrite'])
In [13]: while not it.finished:
    ...:     it[0][...] *= 2
    ...:     arr[it.multi_index] += 100
    ...:     it.iternext()
    ...:     

In [14]: arr       # values are doubled and add by 100
Out[14]: 
array([[100, 104, 108, 112],
       [116, 120, 124, 128],
       [132, 136, 140, 144]])

Without multi_index I could still create an nditer object, and iterate with the while not finished syntax. Instead of accessing x[...] I'd have to use it[0][...].

np.ndindex is a more convenient way of generating a multi_index. Look at its code. It's one of the few numpy functions that uses np.nditer.

In [26]: for idx in np.ndindex(arr.shape):
    ...:     print(idx)
    ...:     arr[idx] -= 100
    ...:     
(0, 0)
(0, 1)
...
(2, 3)
In [27]: arr
Out[27]: 
array([[ 0,  4,  8, 12],
       [16, 20, 24, 28],
       [32, 36, 40, 44]])

But

While it is fun to play with nditer, it isn't practical, at least not in pure Python code. It is most useful as a stepping stone toward using it in cython or pure c code. See the final example of the iteration page.

Tracking Multi-Index and Modifying Values in NumPy

1 Answers1

But