Image Convolution with callback function in python

Question

I want to loop over the pixels of a binary image in python and set the value of a pixel depending on a surrounding neighborhood of pixels. Similar to convolution but I want create a method that sets the value of the center pixel using a custom function rather than normal convolution that sets the center pixel to the arithmetic mean of the neighborhood.

In essence I would like to create a function that does the following:

def convolve(img, conv_function = lambda subImg: np.mean(subImg)):
  newImage = emptyImage
  for nxn_window in img:
    newImage[center_pixel] = conv_function(nxn_window)
  return newImage

At the moment I have a solution but it is very slow:

#B is the structuing array or convolution window/kernel
def convolve(func):
  def wrapper(img, B):
    #get dimensions of img
    length, width = len(img), len(img[0])

    #half width and length of dimensions
    hw = (int)((len(B) - 1) / 2)
    hh = (int)((len(B[0]) - 1) / 2)

    #convert to npArray for fast operations
    B = np.array(B)

    #initialize empty return image
    retVal = np.zeros([length, width])

    #start loop over the values where the convolution window has a neighborhood 
    for row in range(hh, length - hh):
        for pixel in range(hw, width - hw):
            #window as subarray of pixels
            window = [arr[pixel-hh:pixel+hh+1]
                           for arr in img[row-hw:row+hw+1]]
            retVal[row][pixel] = func(window, B)

    return retVal
  return wrapper

with this function as a decorator I then do

# dilation
@convolve
def __add__(img, B):
    return np.mean(np.logical_and(img, B)) > 0

# erosion
@convolve
def __sub__(img, B):
    return np.mean(np.logical_and(img, B)) == 1

Is there a library that provides this type of function or is there a better way I can loop over the image?

Answer 1

Here's an idea: assign each pixel an array with its neighborhood and then simply apply your custom function to the extended image. It'll be fast BUT will consume more memory ( times more memory; if your B.shape is (3, 3) then you'll need 9 times more memory). Try this:

import numpy as np

def convolve2(func):
    def conv(image, kernel):
        """ Apply given filter on an image """
        k = kernel.shape[0]  # which is assumed equal to kernel.shape[1]
        width = k//2  # note that width == 1 for k == 3 but also width == 1 for k == 2
        a = framed(image, width)  # create a frame around an image to compensate for kernel overlap when shifting
        b = np.empty(image.shape + kernel.shape)  # add two more dimensions for each pixel's neighbourhood
        di, dj = image.shape[:2]  # will be used as delta for slicing
        # add the neighbourhood ('kernel size') to each pixel in preparation for the final step 
        # in other words: slide the image along the kernel rather than sliding the kernel along the image
        for i in range(k):
            for j in range(k):
                b[..., i, j] = a[i:i+di, j:j+dj]
        # apply the desired function
        return func(b, kernel)
    return conv
    

def framed(image, width):
    a = np.zeros(np.array(image.shape) + [2 * width, 2 * width])  # only add the frame to the first two dimensions
    a[width:-width, width:-width] = image  # place the image centered inside the frame
    return a

I've used a greyscale image 512x512 pixels and a filter 3x3 for testing:

embossing_kernel = np.array([
    [-2, -1, 0],
    [-1, 1, 1],
    [0, 1, 2]
])

@convolve2
def filter2(img, B):
    return np.sum(img * B, axis=(2,3))

@convolve2
def __add2__(img, B):
    return np.mean(np.logical_and(img, B), axis=(2,3)) > 0

# image_gray is a 2D grayscale image (not color/RGB)
b = filter2(image_gray, embossing_kernel)

To compare with your convolve I've used:

@convolve
def filter(img, B):
    return np.sum(img * B)

@convolve
def __add__(img, B):
    return np.mean(np.logical_and(img, B)) > 0

b = filter2(image_gray, embossing_kernel)

The time for convolve was 4.3 s, for convolve2 0.05 s on my machine.

In my case the custom function needs to specify the axes over which to operate, i.e., the additional dimensions holding the neighborhood data. Perhaps the axes could be avoided too but I haven't tried.

Note: this works for 2D images (grayscale) (as you asked about binary images) but can be easily extended to 3D (color) images. In your case you could probably get rid of the frame (or fill it with zeros or ones e.g., in case of repeated application of the function).

In case memory was an issue you might want to adapt a fast implementation of convolve I've posted here: https://stackoverflow.com/a/74288118/20188124.