Extract multiple windows/patches from an (image) array, as defined in another array

Question

I have an image im which is an array as given by imread. Say e.g.

im = np.array([[1,2,3,4],
               [2,3,4,5],
               [3,4,5,6],
               [4,5,6,7]]

I have another (n,4) array of windows where each row defines a patch of the image as (x, y, w, h). E.g.

windows = np.array([[0,0,2,2],
                    [1,1,2,2]]

I'd like to extract all of these patches from im as sub-arrays without looping through. My current looping solution is something like:

for x, y, w, h in windows:
    patch = im[y:(y+h),x:(x+w)]

But I'd like a nice array-based operation to get all of them, if possible.

Thanks.

But since the images can have different width and height, the shapes are not uniform of the images. — Willem Van Onsem, Nov 21 '17 at 11:25
@Divakar in my case I won't have any windows which extend past the image by construction. But let's say I'd like to just discard any windows that don't fit within the constraints of `im`, so they wouldn't show up in the resulting patches array. — J. Doe, Nov 21 '17 at 11:29
@DanielF yes, though if it simplifies I can deal with it elsewhere and have them be constant in this context. — J. Doe, Nov 21 '17 at 21:27
You can't get a single array as a return then due to uneven dimensions. Your output would be a list of window arrays at best, and then our loopy code is probably best (short of a `@jit` function maybe) — Daniel F, Nov 22 '17 at 06:47
It's actually a very different problem depending on if the window changes. If it doesn't there's a bunch of solutions like `numpy`'s `as_strided`, `skimage`'s `view_as_windows`, or if you want something even more robust my own [`window_nd`](https://stackoverflow.com/questions/45960192/using-numpy-as-strided-function-to-create-patches-tiles-rolling-or-sliding-w) function. — Daniel F, Nov 22 '17 at 06:56
@J.Doe yes, sorry, I've marked one as accepted. Both were fine. I was waiting a bit to see if my comments might prompt some other solution. Thanks. — J. Doe, Nov 22 '17 at 13:46

score 1 · Accepted Answer · answered Nov 21 '17 at 11:33

For same window sizes, we could get views with help from scikit-image's view_as_windows, like so -

from skimage.util.shape import view_as_windows

im4D = view_as_windows(im, (windows[0,2],windows[0,3]))
out = im4D[windows[:,0], windows[:,1]]

Sample run -

In [191]: im4D = view_as_windows(im, (windows[0,2],windows[0,3]))

In [192]: im4D[windows[:,0], windows[:,1]]
Out[192]: 
array([[[1, 2],
        [2, 3]],

       [[3, 4],
        [4, 5]]])

score 0 · Answer 2 · answered Nov 21 '17 at 11:47

If scikit is not available we can homebrew @Divakar's solution with numpy.lib.stride_tricks. The same constraint (all windows must have same shape) applies:

import numpy as np
from numpy.lib.stride_tricks import as_strided

im = np.array([[1,2,3,4],
               [2,3,4,5],
               [3,4,5,6],
               [4,5,6,7]])

windows = np.array([[0,0,2,2],
                    [1,1,2,2]])

Y, X = im.shape
y, x = windows[0, 2:]
cutmeup = as_strided(im, shape=(Y-y+1, X-x+1, y, x), strides=2*im.strides)
print(cutmeup[windows[:, 0], windows[:, 1]])

Output:

[[[1 2]
  [2 3]]

 [[3 4]
  [4 5]]]

Extract multiple windows/patches from an (image) array, as defined in another array

2 Answers2

Linked