5

Resizing an image to 200% yields a difference in quality between Pillow and pyvips.

While Pillow is very accurate in the reproduction, vips exaggerates noise and contrast.

I need to use vips for very large images, but the quality is unsatisfactory. Any idea how to get better upscaling from vips? (From the docs I gathered that upscaling isn't really that important for vips and most thought has gone into downscaling).

example:

from PIL import Image
import pyvips
import numpy as np

#Vips
img = pyvips.Image.new_from_file("mypic.jpg", access='sequential')
out = img.resize(2, kernel = "linear")
out.write_to_file("mypic_vips_resized.tif")

#Pillow
img = np.array(Image.open("mypic.jpg"))
h, w = img.shape[:2]
out = Image.fromarray(img,mode="RGB")
out = out.resize((w*2,h*2), Image.BILINEAR)
out.save("mypic_PIL_resized.tif", format='TIFF', compression='None')

Original:
original
Pillow:
Pillow
Vips:
Vips

Abstract examples (10*10 Pixels)

Original:
Original
Pillow Bilinear:
Pillow Bilinear
Vips linear:
Vips linear

eyllanesc
  • 235,170
  • 19
  • 170
  • 241
Horst
  • 292
  • 2
  • 14
  • 1
    You're using different upsizing methods --- `bilinear` for pillow, and `cubic` for pyvips. Try setting vips to `linear` and they should match. You could experiment with the other pyvips kernels: nearest, linear, cubic, mitchell, lanczos2, lanczos3. – jcupitt Oct 29 '18 at 17:23
  • @jcupitt The image is actually from vips linear (I tried all methods to get the closest match) Just an error in my abbreviated code. – Horst Oct 29 '18 at 18:18
  • I made the code snippet workable, if you save the original as mypic.jpg you should be able to see the difference (However I don't know if SO messes with the uploaded jpg). Anyway, works for any image. – Horst Oct 29 '18 at 18:26
  • 1
    You're right, the libvips `resize` operator is using a simple interpolatory bilinear for upscale, whereas Pillow is using a triangle filter. You can get the effect of a triangle filter by blurring before upsize. I'll make a proper answer. – jcupitt Oct 30 '18 at 13:26

1 Answers1

2

It looks like Pillow is upsizing with a triangle filter for LINEAR, whereas for upsize libvips is doing simple interpolation. libvips uses a triangle filter for downsize.

If you imagine pixels:

A 
B 
C 

Then Pillow is calculating the new pixel between A and B, the pixel at B's position, and the new pixel between B and C as:

(A + B) / 2
(A + B) / 4 + B / 2 + (B + C) / 4
(B + C) / 2

Whereas libvips is calculating:

(A + B) / 2
B
(B + C) / 2

You can get the effect of a triangle filter by doing a very mild blur first. If I change your program to be:

img = pyvips.Image.new_from_file('mypic.png', access='sequential')
img = img.gaussblur(0.45, precision='float', min_ampl=0.01).cast('uchar')
out = img.resize(2, kernel='linear')
out.write_to_file('mypic_vips_resized_blur.png')

ie. do a small radius, high-precision gaussblur first, I get:

enter image description here

Where left-to-right the images are 1) a simple x2 pixel double, 2) Pillow LINEAR, 3) libvips linear, and 4) libvips gaussblur + linear. You'll probably need to click on the image or your browser will shrink it and blur it.

2) and 4) seem reasonably close to my (not great) eyes. 3) seems arguably more truthful to the original, since the ringing and noise present in the original have not been smoothed out.

jcupitt
  • 10,213
  • 2
  • 23
  • 39
  • Thanks a lot for the detailed explanation. I've tried the suggested gaussblur and it did indeed improve the results. Still I find the Pillow verion to be more pleasing even if the reproduction is not as faithful for a single image. However since I'm working with image stacks of ~15 images and calculate average and median, the output of pillow does look better to me (less noise and better contrast compared with gauss-vips). Is there a chance that pillows variant of the linear filter would be implemented in libvips? I know upscaling is not a priority, but would still fit pillows original goals. – Horst Oct 30 '18 at 17:30
  • In the last sentence I mean vips' original goals of course. – Horst Oct 30 '18 at 18:22
  • Sure, open an issue on the libvips tracker and we can tag it as an enhancement. Could you explain your whole pipeline? Perhaps the blur could be added at some other point. – jcupitt Oct 30 '18 at 22:27
  • Great, will do! My pipeline: Use several handheld burst-shots with the aim of getting image information beyond the bayer filter in a camera -> scale to 200% -> align (hugin) -> calculate average/median for each pixel in the stack. The result is a picture with a higher resolution in the sense of resolving finer detail as the slight differences in recording angle change the objects position of edges on the bayer filter and thus we can in theory gain full advertised nMP resolution as opposed to extrapolated nMP. Pentax introduced this as pixel-shift moving only the sensor precisely. – Horst Oct 30 '18 at 22:35
  • Ah OK, yes, I've worked on things like that. I would scale to 200% using libvips interpolation -- you don't want to do any extra smoothing until the end of processing. I think you'll find avg / median adds quite enough. libvips has an operator for rank filter on an image stack https://libvips.github.io/libvips/API/current/libvips-conversion.html#vips-bandrank – jcupitt Oct 31 '18 at 08:12
  • Bandrank works brilliantly! Do you have any advice on using bandmean? I thought it behaved the same as bandrank which it doesn't. If I can get a method to average the images in vips I could drop Pillow/Skimage altogether. I've tried to find documentation, but I'm having trouble finding anything for averaging. bandmean does not accept lists of images as bandrank does so I'm at a bit of a loss. – Horst Nov 03 '18 at 12:04
  • `bandmean` finds the average across the image bands https://libvips.github.io/libvips/API/current/libvips-conversion.html#vips-bandmean If you want to average a set of images, just sum them and divide. There's also `sum` to quickly sum an array of images https://libvips.github.io/libvips/API/current/libvips-arithmetic.html#vips-sum so eg. `pyvips.Image.sum(imagearray) / len(imagearray)`. – jcupitt Nov 03 '18 at 12:54
  • Oh, pyvips docs https://libvips.github.io/pyvips/vimage.html#pyvips.Image.bandmean and https://libvips.github.io/pyvips/vimage.html#pyvips.Image.sum – jcupitt Nov 03 '18 at 12:55
  • Sum, of course. Git a bit caught up in fancy functions. For anyone else reading this, using pyvips rint and cast afterwards allows conversion to int-based images opposed to the float output from the division. – Horst Nov 05 '18 at 14:46