Indices of fixed size sub-matrices of numpy array

Question

I am implementing an algorithm which requires me to look at non-overlapping consecutive submatrices within a (strictly two dimensional) numpy array. eg, for the 12 by 12

>>> a = np.random.randint(20, size=(12, 12)); a
array([[ 4,  0, 12, 14,  3,  8, 14, 12, 11, 18,  6,  6],
       [15, 13,  2, 18, 15, 15, 16,  2,  9, 16,  6,  4],
       [18, 18,  3,  8,  1, 15, 14, 13, 13, 13,  7,  0],
       [ 1,  9,  3,  6,  0,  4,  3, 15,  0,  9, 11, 12],
       [ 5, 15,  5,  6,  4,  4, 18, 13, 10, 17, 11,  8],
       [13, 17,  8, 15, 17, 12,  7,  1, 13, 15,  0, 18],
       [ 2,  1, 11, 12,  3, 16, 11,  9, 10, 15,  4, 16],
       [19, 11, 10,  7, 10, 19,  7, 13, 11,  9, 17,  8],
       [14, 14, 17,  0,  0,  0, 11,  1, 10, 14,  2,  7],
       [ 6, 15,  6,  7, 15, 19,  2,  4,  6, 16,  0,  3],
       [ 5, 10,  7,  5,  0,  8,  5,  8,  9, 14,  4,  3],
       [17,  2,  0,  3, 15, 10, 14,  1,  0,  7, 16,  2]])

and looking at 3x3 submatrices, I would want the first 3x3 submatrix to be from the upper left corner:

>>> a[0:3, 0:3]
array([[ 4,  0, 12],
       [15, 13,  2],
       [18, 18,  3]])

The next along to be given by a[0:3, 3:6] and so on. It doesn't matter if the last such set of indices in each row or column runs off the end of the array - numpy's behavior of simply giving the portion within the slice that exists is sufficient.

I want a way to generate these slice indices programatically for arbitrarily sized matrices and submatrices. I currently have this:

size = 3
x_max = a.shape[0]
xcoords = range(0, x_max, size)
xcoords = zip(xcoords, xcoords[1:])

and similarly to generate y_coords, so that the series of indices is given by itertools.product(xcoords, ycoords).

My question is: is there a more direct way to do this, perhaps using numpy.mgrid or some other numpy technique?

Answer 1

Getting the indexes

Here's a quick way to get a specific size x size block:

base = np.arange(size) # Just the base set of indexes
row = 1                # Which block you want
col = 0                
block = a[base[:, np.newaxis] + row * size, base + col * size]

If you wanted you could build up matrices similar to your xcoords like:

y, x = np.mgrid[0:a.shape[0]/size, 0:a.shape[1]/size]
y_coords = y[..., np.newaxis] * size + base
x_coords = x[..., np.newaxis] * size + base

Then you could access a block like this:

block = a[y_coords[row, col][:, np.newaxis], x_coords[row, col]]

---

Getting the blocks directly

If you just want to get the blocks (and not the indexes of the block entries), I'd use np.split (twice):

blocks = map(lambda x : np.split(x, a.shape[1]/size, 1), # Split the columns
                        np.split(a, a.shape[0]/size, 0)) # Split the rows

then you have a 2D list of size x size blocks:

>>> blocks[0][0]
array([[ 4,  0, 12],
       [15, 13,  2],
       [18, 18,  3]])

>>> blocks[1][0]
array([[ 1,  9,  3],
       [ 5, 15,  5],
       [13, 17,  8]])

You could then make this a numpy array and use the same indexing style as above:

>>> blocks = np.array(blocks)
>>> blocks.shape
(4, 4, 3, 3)

Answer 2

You can use the one-liner:

r = 3
c = 3
lenr = a.shape[0]/r
lenc = a.shape[1]/c
np.array([a[i*r:(i+1)*r,j*c:(j+1)*c] for (i,j) in np.ndindex(lenr,lenc)]).reshape(lenr,lenc,r,c)

Answer 3

I'm adding this answer to an old question since an edit has bumped this question up. Here's an alternative way to calculate blocks:

size = 3
lenr, lenc = int(a.shape[0]/size), int(a.shape[1]/size)

t = a.reshape(lenr,size,lenc,size).transpose(0, 2, 1, 3)

Profiling shows that this is the fastest. Profiling done with python 3.5, and the results from map passed to array() for compatibility, since in 3.5 map returns an iterator.

reshape/transpose:   643 ns per loop
reshape/index:       45.8 µs per loop
Map/split:           10.3 µs per loop

It's interesting that the iterator version of map is faster. In any case, using reshape and transpose is fastest.