59

I have a numpy array that I wish to resize using opencv. Its values range from 0 to 255. If I opt to use cv2.INTER_CUBIC, I may get values outside this range. This is undesirable, since the resized array is supposed to still represent an image. One solution is to clip the results to [0, 255]. Another is to use a different interpolation method. It is my understanding that using INTER_AREA is valid for down-sampling an image, but works similar to nearest neighbor for upsampling it, rendering it less than optimal for my purpose.

Should I use INTER_CUBIC (and clip), INTER_AREA, or INTER_LINEAR?

an example for values outside of range using INTER_CUBIC:

a = np.array( [ 0, 10, 20, 0, 5, 2, 255, 0, 255 ] ).reshape( ( 3, 3 ) )
[[  0  10  20]
 [  0   5   2]
 [255   0 255]]

b = cv2.resize( a.astype('float'), ( 4, 4 ), interpolation = cv2.INTER_CUBIC )
[[   0.            5.42489886   15.43670964   21.29199219]
 [ -28.01513672   -2.46422291    1.62949324  -19.30908203]
 [  91.88964844   25.07939219   24.75106835   91.19140625]
 [ 273.30322266   68.20603609   68.13853455  273.15966797]]

Edit: As berak pointed out, converting the type to float (from int64) allows for values outside the original range. the cv2.resize() function does not work with the default 'int64' type. However, converting to 'uint8' will automatically saturate the values to [0..255].

Also, as pointed out by SaulloCastro, another related answer demonstrated scipy's interpolation, and that there the defualt method is the cubic interpolation (with saturation).

Roee E
  • 753
  • 1
  • 6
  • 7
  • why a.astype('float') ? without it it saturates properly – berak May 25 '14 at 09:14
  • You are right, it does. I was concerned, however, that I might be misusing the cubic interpolation method because of the negative values I had encountered. Note, also, that the input array "a" is only 3x3, which is too small for the INTER_CUBIC's 4x4 patch. But This phenomenon also occurs with larger images. – Roee E May 25 '14 at 11:15
  • 1
    What does "best" in your context mean? Since you have no information but the mean of the block represented by the new subpixels, you seem to have a model in mind how they should look like. – Bort May 25 '14 at 11:26
  • 1
    That is a good question. Since I want to resize an image to a larger size, I want an interpolation method that minimizes artifacts and/or aliasing. However, I am not entirely sure what is the best practice, or what I should look for when resizing an image. I guess different interpolation methods are "best" in different aspects, and so that was part of my question. – Roee E May 25 '14 at 13:15
  • @RoeeE [this answer](http://stackoverflow.com/a/16510074/832621) will probably help you.. – Saullo G. P. Castro May 26 '14 at 10:19
  • @SaulloCastro Thanks, it does help. Specifically they point out that cubic interpolation is the default for scipy.ndimage.zoom. So, I deduct that using the cubic interpolation with the 'uint8' dtype is ok for my purposes (as pointed out by berak above). – Roee E May 26 '14 at 10:43
  • For shrinking, how exactly does INTER_AREA interpolation work ? – seralouk Jun 03 '19 at 13:00
  • Much more flexible interpolation is possible in ImageMagick so that higher quality results can be achieved -- mitigate artifacts vs sharper results. Some of which are done in linear color space. See https://imagemagick.org/Usage/filter/ and https://imagemagick.org/Usage/filter/nicolas/ – fmw42 Aug 15 '19 at 04:55
  • Bicubic for enlargement, bilinear for reduction (https://hammadiqbal.com/tutorial/the-best-method-of-image-reduction-in-photoshop/) – rinogo Mar 31 '21 at 18:09

4 Answers4

74

If you are enlarging the image, you should prefer to use INTER_LINEAR or INTER_CUBIC interpolation. If you are shrinking the image, you should prefer to use INTER_AREA interpolation.

Cubic interpolation is computationally more complex, and hence slower than linear interpolation. However, the quality of the resulting image will be higher.

Shivam Kushwaha
  • 990
  • 8
  • 12
  • 4
    For shrinking, how exactly does INTER_AREA interpolation work ? – seralouk Jun 03 '19 at 13:00
  • 29
    Do you have any evidence supporting your claim? – jdhao May 14 '20 at 07:34
  • Related reading: https://hammadiqbal.com/tutorial/the-best-method-of-image-reduction-in-photoshop/ – rinogo Mar 31 '21 at 18:09
  • Also https://chadrick-kwag.net/cv2-resize-interpolation-methods/ – rinogo Mar 31 '21 at 18:12
  • 5
    @jdhao From the [docs](https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html#ga47a974309e9102f5f08231edc7e7529d): To shrink an image, it will generally look best with `INTER_AREA` interpolation, whereas to enlarge an image, it will generally look best with `INTER_CUBIC` (slow) or `INTER_LINEAR` (faster but still looks OK). – Salvatore May 13 '22 at 21:24
  • 1
    @seralouk A full list of interpolation methods and descriptions can also be found [here](https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html#ga5bb5a1fea74ea38e1a5445ca803ff121). – Salvatore May 13 '22 at 21:28
21

To overcome such problem you should find out the new size of the given image where the interpolation can be made. And copy interpolated sampled image on the target image like:

# create target image and copy sample image into it
(wt, ht) = imgSize # target image size
(h, w) = img.shape # given image size
fx = w / wt
fy = h / ht
f = max(fx, fy)
newSize = (max(min(wt, int(w / f)), 1),
           max(min(ht, int(h / f)), 1))  # scale according to f (result at least 1 and at most wt or ht)
img = cv2.resize(img, newSize, interpolation=cv2.INTER_CUBIC) #INTER_CUBIC interpolation
target = np.ones([ht, wt]) * 255  # shape=(64,800)
target[0:newSize[1], 0:newSize[0]] = img

Some of the possible interpolation in openCV are:

  • INTER_NEAREST – a nearest-neighbor interpolation
  • INTER_LINEAR – a bilinear interpolation (used by default)
  • INTER_AREA – resampling using pixel area relation. It may be a preferred method for image decimation, as it gives moire’-free results. But when the image is zoomed, it is similar to the INTER_NEAREST method.
  • INTER_CUBIC – a bicubic interpolation over 4×4 pixel neighborhood
  • INTER_LANCZOS4 – a Lanczos interpolation over 8×8 pixel neighborhood

See here for results in each interpolation.

susan097
  • 3,500
  • 1
  • 23
  • 30
5

I think you should start with INTER_LINEAR which is the default option for resize() function. It combines sufficiently good visual results with sufficiently good time performance (although it is not as fast as INTER_NEAREST). And it won't create those out-of-range values.

Michael Burdinov
  • 4,348
  • 1
  • 17
  • 28
4

My this answer is based on testing. And in the end it supports the answer of @shivam. I tested on these interpolation method, for both combination, shrinking and enlarging. And after enlarging I calculated psnr with orignal image.

[cv2.INTER_AREA,
cv2.INTER_BITS,
cv2.INTER_BITS2,
cv2.INTER_CUBIC,
cv2.INTER_LANCZOS4,
cv2.INTER_LINEAR,
cv2.INTER_LINEAR_EXACT,
cv2.INTER_NEAREST]

shirking=0.25
enlarge=4

Tested this on 165 images of different shapes. For results I picked maximum and 2nd maximum psnr and calculated the count. For the Maximum the count for interpolation is show in image below. maximum psnr

From this test the maximum psnr is given by the combination of AREA and LANCZOS4 which gave max psnr for 141/204 images.

I also wanted to include the 2nd maximum. So here are the results of 2nd Maximum only. 2nd max psnr

Here the AREA and CUBIC gave the 2nd best result. 19/204 has the highest psnr and 158/347 have the 2nd highes psnr using AREA + CUBIC.

This results were vague so I opened the files for which CUBIC gave the highes psnr. Turns out the images with lot of texture/abstraction gave highest psnr using CUBIC.

So I did furthure tests only for AREA+CUBIC and AREA+LANCZOS4. I came to the conclusion that if you are shrinking image <10 times then go for the LANCZOS4. It will give you better results for less than 10times zoom and if the image is large than it's better than CUBIC.

As for my program I was shiriking image 4 times so for me the AREA+LANCZOS4 works better.

Scripts and images: https://github.com/crackaf/triple-recovery/tree/main/tests

crackaf
  • 492
  • 2
  • 11