Python: perform blur only within a mask of image

Question

I have a greyscale image and a binary mask of an ROI in that image. I would like to perform a blur operation on the greyscale image but only within the confines of the mask. Right now I'm blurring the whole image and than just removing items outside the mask, but I don't want pixels outside of the mask affecting my ROI. Is there a way to do this without building a custom blur function?

hoping for something like:

import scipy
blurredImage = scipy.ndimage.filters.gaussian_filter(img, sigma = 3, weight = myMask)

@stefan:

blur = 3
invmask = np.logical_not(mask).astype(int)

masked = img * mask
remaining = img * invmask

blurred = scipy.ndimage.filters.gaussian_filter(masked, sigma = blur)
blurred = blurred+remaining

Dilate approach:

blur = 3
invmask = np.logical_not(mask).astype(int)    
masked = img * mask
masked2 = scipy.ndimage.morphology.grey_dilation(masked,size=(5,5))
masked2 = masked2 *invmask
masked2 = masked + masked2
blurred = scipy.ndimage.filters.gaussian_filter(masked2, sigma = blur)

The mask multiplied with the image only contains the pixels inside the roi. The inverse of the mask multiplied with the image is the remaining pixels. So blur the masked pixels, and add the remaining pixels. — Stefan, Jan 10 '20 at 16:13
You could extract (by slicing) the ROI that you want to blur and then have the *"virtual pixels"* you need around the edge be synthetically generated as repeating the content inside the ROI as you get further away. — Mark Setchell, Jan 10 '20 at 16:13
@stefan, thanks i've added the code for how i understand your comment. Im probably misunderstanding but this would be similar to my current approach except the edges of my ROI would be blurred against the 0 background.. can you clarify? — user3470496, Jan 10 '20 at 16:43
@mark, thanks Mark, I had a similar thought but wasn't able to find any kind of greyscale dilate function, is there a way to do this without a messy custom function? — user3470496, Jan 10 '20 at 16:44
Does this answer your question? [How to apply a uniform filter using SciPy Image where the data outside the boundary is not tallied?](https://stackoverflow.com/questions/53822897/how-to-apply-a-uniform-filter-using-scipy-image-where-the-data-outside-the-bound) — Cris Luengo, Jan 10 '20 at 16:47
Actually, I just realized that the linked answer doesn't fully answer your question. I'll complete it below. — Cris Luengo, Jan 10 '20 at 16:52

score 8 · Accepted Answer · edited Jul 23 '23 at 19:56

The right approach to apply a linear filter to a limited domain is to use Normalized Convolution. This method computes (weighted) means within each neighborhood, then normalizes by the (weighted) number of pixels present in that neighborhood. It does so using only two applications of the filter and some trivial per-pixel operations:

# normalized convolution of image with mask
filter = scipy.ndimage.gaussian_filter(img * mask, sigma = blur)
weights = scipy.ndimage.gaussian_filter(mask, sigma = blur)
filter /= weights
# after normalized convolution, you can choose to delete any data outside the mask:
filter *= mask

Note that mask doesn't need to be just 0 and 1, it can contain intermediate values indicating how "certain" you are of the correctness of that pixel's value. But typically it's just 0 for "missing data" and 1 for available data.

gaussian_filter must do its computations in a floating-point format and return an floating-point-valued image. Integer operations will not do the correct thing here.

Here's an example:

2nd image: Plain filtering, then removing the stuff outside the mask. This shows that the data outside the mask influences the result of the filtering.
3rd image: Plain filtering, but setting stuff outside the mask to zero first. This shows that the zeros outside the mask influence the result of the filtering.
4th image: Using normalized convolution: the data outside the masked area does not affect the filtering at all.

This gave me an odd output with most of the masked ROI eroded, maybe im applying wrong? — user3470496, Jan 10 '20 at 17:20
@user3470496: Make note of the last paragraph: you **must** use floating-point operations. The safest best is to convert `mask` to a floating-point array. — Cris Luengo, Jan 10 '20 at 17:23
it worked! got some nans in there that made the output look funky but "filter = np.nan_to_num(filter,nan=0.0)" remedied. Thanks! — user3470496, Jan 17 '20 at 15:47
@user3470496: To avoid NaNs, you could also do `filter /= weights + 0.0001`. — Cris Luengo, Jan 17 '20 at 15:49

score 2 · Answer 2 · answered Jan 10 '20 at 16:56

What you are willing to achieve is uneasy and less well defined than you think.

A blur operation corresponds to some (isotropic) averaging of the pixels in a neighborhood of every pixel. But near the boundaries of the domain, the neighborhoods are incomplete and you need to fix that either

by locally redefining the filter coefficients to avoid "querying" outer pixels; a simple way is to set the filter weights to zero on these pixels; or
by extrapolating the pixel values outside the ROI. This should be done in a way that avoids discontinuities along the boundary, for instance by means of Poisson reconstruction.

For the first approach, you can hijack the standard filter as follows:

set the outer pixels to 0 and blur the whole image; when the filter straddles the outline, you will get a combination of the inner weights and inner pixels only;
set the inner pixels to 1 and blur the image; you will get the sum of the inner weights only;
take the ratio of both blurred images so that the weights get renormalized correctly.

If the images are of an integer type, use 255 instead of 1 to keep sufficient accuracy. Beware that outside the ROI, the ratios will have a zero denominator.

Python: perform blur only within a mask of image

2 Answers2

Linked