fastest way to select 7*7 neighbor pixels for every pixel in an image in Python

Question

need to read an image as an array and for each pixel select 7*7 neighbor pixels then reshape it and put as a first row of training set:

  import numpy as np
  from scipy import misc
  face1=misc.imread('face1.jpg')

face1 dimensions are (288, 352, 3) , need to find 7*7 neighbor pixels for every pixel , so 49*3 color then reshape it as a (1,147) array and stack it into an array for all pixels , i took the following approach:

X_training=np.zeros([1,147] ,dtype=np.uint8)
for i in range(3, face1.shape[0]-3):
    for j in range(3, face1.shape[1]-3):
        block=face1[i-3:i+4,j-3:j+4]
        pxl=np.reshape(block,(1,147))
        X_training=np.vstack((pxl,X_training))

resulting X_training shape is (97572, 147)

and as last row contains all zeros then:

a = len(X_training)-1
X_training = X_training[:a]

above code works well for one picture but with Wall time: 5min 19s i have 2000 images, so it will take ages to do it for all the images. I am looking for a faster way to iterate over every pixel and do the above task.

Edit: this is what i mean by neighbor pixels , for every pixel face1[i-3 : i+4 ,j-3:j+4]

could you break the image into chunks and work different sections with threads or something along those lines? — Jacobr365, Jul 26 '17 at 13:05
I do not really understand which neighboring pixels you want to select. Can you elaborate on that? — jotasi, Jul 26 '17 at 13:06
Think it should be : `X_training=np.zeros([0,147] ,dtype=np.uint8)`. — Divakar, Jul 26 '17 at 13:13
@2cynykyl that is why i started from 3 and end at face1.shape[0]-3 — chessosapiens, Jul 26 '17 at 13:15
@sanaz It has do with confusing people, because you would have the last row of output as all zeros and as such that row doesn't deal with selecting neighbours. People need to understand what you are doing. — Divakar, Jul 26 '17 at 13:18
@Divakar I made an edit, but as the title of question and explanation says i am looking for a way to faster the process, even if one row is all zero. — chessosapiens, Jul 26 '17 at 13:28
Well the title says - `to select 7*7 neighbor pixels for every pixel ..`. With that all zeros row, it might confuse people. Apologies if my tone sounded too picky, just trying to be see a good question here. — Divakar, Jul 26 '17 at 13:30
this might help: https://stackoverflow.com/questions/29925684/generalize-stacking-of-array-elements-neighbors-into-3-d-array — Paul H, Jul 26 '17 at 16:15
@Divakar any idea how can i do exactly same task in pyspark? https://stackoverflow.com/questions/45400235/fastest-way-to-select-77-neighbor-pixels-for-every-pixel-in-an-image-in-pyspark — chessosapiens, Jul 30 '17 at 13:57

MB-F · Accepted Answer · 2017-07-26T14:01:36.333

6

An efficient way is to use stride_tricks to create a 2d rolling window over the image, then flatten it out:

import numpy as np

face1 = np.arange(288*352*3).reshape(288, 352, 3)  # toy data

n = 7  # neighborhood size

h, w, d = face1.shape
s = face1.strides

tmp = np.lib.stride_tricks.as_strided(face1, strides=s[:2] + s,
                                      shape=(h - n + 1, w - n + 1, n, n, d))
X_training = tmp.reshape(-1, n**2 * d)
X_training = X_training[::-1]  # to get the rows into same order as in the question

tmp is a 5D view into the image, where tmp[x, y, :, :, c] is equivalent to the neigborhood face1[x:x+n, y:y+n, c] in color channel c.

edited Jul 26 '17 at 14:01

answered Jul 26 '17 at 13:42

MB-F

22,770
4
61
116

Good one! To replicate the output from OP, we would need a flipping there along rows. – Divakar Jul 26 '17 at 13:57
@Right, Thank you. I've added this as an explicit step because I'm not very fond of negative strides :) – MB-F Jul 26 '17 at 14:02
I have never heard of strides before, but after a bit of reading I see that I've been missing out on some powerful stuff...thanks! – 2cynykyl Jul 27 '17 at 02:04
@2cynykyl good to hear.. I also learned about strides on this site. – MB-F Jul 27 '17 at 06:06
@kazemakase any idea how can i do exactly same task in pyspark? https://stackoverflow.com/questions/45400235/fastest-way-to-select-77-neighbor-pixels-for-every-pixel-in-an-image-in-pyspark – chessosapiens Jul 30 '17 at 13:58
@sanaz no, sorry. I do not know anything about pyspark. – MB-F Jul 30 '17 at 18:38

2cynykyl · Answer 2 · 2017-07-31T14:25:38.697

3

The following is < 1s on my laptop:

import scipy as sp
im = sp.rand(300, 300, 3)

size = 3
ij = sp.meshgrid(range(size, im.shape[0]-size), range(size, im.shape[1]-size))
i = ij[0].T.flatten()
j = ij[1].T.flatten()

N = len(i)
L = (2*size + 1)**2
X_training = sp.empty(shape=[N, 3*L])

for pixel in range(N):
    si = (slice(i[pixel]-size, i[pixel]+size+1))
    sj = (slice(j[pixel]-size, j[pixel]+size+1))
    X_training[pixel, :] = im[si, sj, :].flatten()

X_training = X_training[-1::-1, :]

I'm always a bit sad when I can't think of one-line vectorized version, but at least it's faster for you.

edited Jul 31 '17 at 14:25

answered Jul 26 '17 at 13:42

2cynykyl

983
9
12

can you compare your answer with best answer? it does not return the same result. there should be some mistake somewhere , my own approach and best answer return exactly same result – chessosapiens Jul 30 '17 at 12:59
I've updated my answer, and it now produces the same output as @kazemakase. The rows were were in a different ordering, so I transposed the ``ij`` array, and I also reversed the overall order of the rows in the final answer. – 2cynykyl Jul 31 '17 at 14:27

score 3 · Answer 3 · answered Aug 02 '17 at 20:41

3

Using scikit-image:

import numpy as np
from skimage import util

image = np.random.random((288, 352, 3))
windows = util.view_as_windows(image, (7, 7, 3))

out = windows.reshape(-1, 7 * 7 * 3)

answered Aug 02 '17 at 20:41

Stefan van der Walt

7,165
1
32
41

+1 for using skimage...but I just read the docs for this ``view_as_window`` function and it's has terrible performance issues! Someone should really re-implement it using the ``stride_tricks`` in the accepted answer. – 2cynykyl Aug 25 '17 at 21:02
I'm not sure what you mean; it *does* use strides: https://github.com/scikit-image/scikit-image/blob/master/skimage/util/shape.py#L107 – Stefan van der Walt Aug 26 '17 at 16:56
my apologies for not checking source...but the docstring really makes it sounds like it is creating new copies of the array, and that memory can get explosively large. My understanding of strides is that it provides new 'views' onto a single copy of the underlying data, hence no copies and much faster. How does your answer compare to @kazemakase in terms of speed I wonder? – 2cynykyl Aug 27 '17 at 18:25
It'd be great if you could compare! We'd be very happy for PRs to improve the documentation or implementation. – Stefan van der Walt Aug 27 '17 at 21:33

fastest way to select 7*7 neighbor pixels for every pixel in an image in Python

3 Answers3