4

A toy-case for my problem:

I have a numpy array of size, say, 1000:

import numpy as np
a = np.arange(1000)

I also have a "projection array" p which is a mapping from a to another array b:

p = np.random.randint(0,1000,(1000,1000))

It is easy to get b from a using "fancy indexing":

b = a[p]

But b is not a view, as noted by several previous questions/answers and the numpy documentation.

Unfortunately, in my case only the values in a change over the course of a long simulation and using fancy indexing at each iteration to obtain b becomes very costly. I only read from b and do not modify it.

I understand it is not possible (yet) to solve this with fancy indexing.

I was wondering if anyone had a similar problem/bottleneck and came up with some other workaround?

upperBound
  • 680
  • 4
  • 11
  • 1
    Is all of `b` accessed in every iteration? If yes, there is no cheaper way than extracting all of `b` – you might need to improve the algorithm in other ways. If no, you don't need to build `b`, and instead of accessing `b[i, j]`, you can use `a[p[i, j]]`. – Sven Marnach Apr 04 '12 at 21:19

1 Answers1

2

What your asking for isn't practical and that's why the numpy folks haven't implemented it. You could do it yourself with something like:

class FancyView(object):
    def __init__(self, array, index):
        self._array = array
        self._index = index.copy()
    def __array__(self):
        return self._array[self._index]
    def __getitem__(self, index):
        return self._array[self._index[index]]

b = FancyView(a, p)

But notice that the expensive a[p] operation will get called every time you use b as an array. There is no other practice way of making a 'view' of this kind. Numpy can get away with using views for basic slicing because it can manipulate the strides, but there is no way to do something like this using strides.

If you only need parts of b you might be able to get some time savings by indexing the fancy view instead of using it as an array.

Bi Rico
  • 25,283
  • 3
  • 52
  • 75