Efficiently subset a 2D numpy array iteratively

Question

I have a large image on which I want to perform an operation from a moving window over the whole window. Here is a reproducible example:

Given an array named image with shape (5, 5), I need to extract subsets from the array in 3x3 windows.

import numpy as np

# example dada
image = np.array([[1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5], [1,2,3,4,5]])

Out[1]:

array([[1, 2, 3, 4, 5],
       [1, 2, 3, 4, 5],
       [1, 2, 3, 4, 5],
       [1, 2, 3, 4, 5],
       [1, 2, 3, 4, 5]])

For a 3x3 window the first subset is:

# first iteration
window_size = 3
image[0:window_size, 0:window_size] # from 1st to 3th row and from 1st to 3th col

Out[2]:

array([[1, 2, 3],
       [1, 2, 3],
       [1, 2, 3]])

So I can access to the different subsets using a nested loop:

for i in range(0, image.shape[0]-window_size+1):
   for j in range(0, image.shape[1]-window_size+1):
      a = (j,i)     # top left value
      b = (j,i+window_size)   # top right value
      c = (j+window_size,i)   # bottom left value
      d = (j+window_size,i+window_size) # bottom right value
      print('Window position', a,b,c,d)
      subset = image[i:i+window_size, j:j+window_size]
      print(subset)

Is there a more efficient way to perform this operation and avoid doing these two loops?

https://numpy.org/devdocs/reference/generated/numpy.lib.stride_tricks.sliding_window_view.html — I don’t know if this is much faster, the docs do warn about it being slower than specialized solutions like the convolution. — Cris Luengo, Jul 05 '23 at 14:27
You can use `np.lib.stride_tricks.sliding_window_view(image, (3, 3))`, but it depends what you want to do with that… — mozway, Jul 05 '23 at 14:28
Btw, your code for slicing is incorrect, this should be: `image[i:i+window_size, j:j+window_size]` — mozway, Jul 05 '23 at 14:32
Your approach is what I am waiting for. Thank you very much. PS: I fixed the error in the reproducible code. — sermomon, Jul 05 '23 at 14:38
There are several other questions about vectorizing moving windows on arrays that you should reference, [here](https://stackoverflow.com/questions/15722324/sliding-window-of-m-by-n-shape-numpy-ndarray), [here](https://stackoverflow.com/questions/8174467/vectorized-moving-window-on-2d-array-in-numpy), and [here](https://stackoverflow.com/questions/48215914/vectorized-2-d-moving-window-in-numpy-including-edges) — astroChance, Jul 05 '23 at 14:56
I didn't know you can use tuples to index in this way, so I tried running you code. As I expected, `image[a:b,a:c]` doesn't work. That should probably be `image[a[0]:d[0],a[1]:d[1]]`. — Cris Luengo, Jul 05 '23 at 15:23

sermomon · Accepted Answer · 2023-07-06T21:11:01.660

Thanks to @CrisLuengo and @mozway comments I can answer my own question. Indeed the dedicated function np.lib.stride_tricks.sliding_window_view is more efficient than a nested loop. Here is a small timing test with a window size of 3x3 for arias of 100x100, 1000x1000 and 10000x10000.

import numpy as np
import time

image = np.random.random((10000, 10000))
window_size = 3

# nested loop
start = time.time()
subsets = []
for i in range(0, image.shape[0]-window_size+1):
   for j in range(0, image.shape[1]-window_size+1):
      subset = image[i:i+window_size, j:j+window_size]
      subsets.append(subset)
end = time.time()
print(end - start)

# dedicated function
start = time.time()
subset = np.lib.stride_tricks.sliding_window_view(image, (window_size, window_size))
subsets = subset.reshape(subset.shape[0]*subset.shape[1], window_size, window_size)
end = time.time()
print(end - start)

However, there may be an even more efficient solution using scipy.ndimage.generic_filter. This function allows to apply a user-defined function to the different positions of the window. If I find a better solution with this approach I will update my answer.

Consider using `time.perf_counter()` if you are trying to measure performance of approaches — astroChance, Jul 06 '23 at 18:16

score -2 · Answer 2 · edited Jul 05 '23 at 15:24

-2

Try these steps:

import numpy as np

# example data
image = np.array([[1, 2, 3, 4, 5],
                  [1, 2, 3, 4, 5],
                  [1, 2, 3, 4, 5],
                  [1, 2, 3, 4, 5],
                  [1, 2, 3, 4, 5]])

# Define the window size
window_size = 3

# Calculate the number of rows and columns in the output array
output_shape = (image.shape[0] - window_size + 1, image.shape[1] - 
window_size + 1)

# Create an empty array to store the subsets
subsets = np.empty(output_shape, dtype=object)

# Extract subsets using nested loops
for i in range(output_shape[0]):
    for j in range(output_shape[1]):
        subsets[i, j] = image[i:i + window_size, j:j + window_size]

# Print the extracted subsets
for i in range(output_shape[0]):
    for j in range(output_shape[1]):
        print(f"Subset at position ({i}, {j}):\n{subsets[i, j]}\n")

edited Jul 05 '23 at 15:24

Cris Luengo

55,762
10
62
120

answered Jul 05 '23 at 14:43

masoudseddighi

33
1
6

7

This answer would be more helpful if it explained how it answers the question -- code-only answers are never as useful as answers that explain the solution. That said, the question was "*Is there a more efficient way to perform this operation and avoid doing these two loops?*", so code that uses loops is probably not answering the question. – Cris Luengo Jul 05 '23 at 15:27
3

Can you explain how your answer is so similar to the one generated by ChatGPT? – Sebastian Wozny Jul 05 '23 at 18:01
how is this better? – juanpa.arrivillaga Jul 06 '23 at 21:22

Efficiently subset a 2D numpy array iteratively

2 Answers2