Bilinear upsample in tensorflow?

Question

I want to do a simple bilinear resize (not necessarily by an integer factor) in TensorFlow. For example, starting from a (32,3,64,64) tensor, I would like a (32,3,96,96) tensor, where each 64x64 has been rescaled by a factor of 1.5 using bilinear interpolation. What is the best way to do that?

I would like this to support arbitrary factors > 1, not just 1.5 specifically.

Note: the operation on each 64x64 would be the same as what skimage.transform.rescale (scale=1.5, order=1) does.

score 15 · Accepted Answer · edited Nov 02 '18 at 09:31

15

tf.image.resize_images should do what you need. It accepts both 3d (single image) and 4d (batch of images) tensors, with arbitrary depth (number of channels). So this should hopefully work:

# it's height, width in TF - not width, height
new_height = int(round(old_height * scale))
new_width = int(round(old_width * scale))
resized = tf.image.resize_images(input_tensor, [new_height, new_width])

Bilinear interpolation is the default so you don't need to specify it. You could also use resize_bilinear directly.

edited Nov 02 '18 at 09:31

laura_b

119
8

answered Apr 19 '16 at 22:32

etarion

16,935
4
43
66

4

Thanks, that's just what I was looking for. I wonder, is it possible to backpropagate through resize_images? – Alex I Apr 22 '16 at 08:18
@AlexI is is possible to do this? Backpropogate through the resize? – mattdns Aug 03 '16 at 11:20
4

@mattdns Yes, absolutely. You can backprop through resize as it's a linear operator. Tensorflow does this with no problem. – RawMean Nov 23 '16 at 23:11
@RawMean Don't edit bugs in - before r0.10 height and width are different arguments, look at the docs – etarion Nov 24 '16 at 15:59
@etarion It might have been, but it's not anymore. Do you really want your answer refer to a deprecated API that does not compile anymore? – RawMean Nov 25 '16 at 01:00
1

@RawMean r0.11 was released 15 days ago, not everybody has updated yet. And the answer clearly mentions both versions. – etarion Nov 25 '16 at 09:36

score 7 · Answer 2 · edited Jun 20 '20 at 09:12

7

I would suggest not to use any of the tf.image.resize_* functions as they suffer from a nasty bug that will not be fixed.

A new, different set of image resampling functions is apparently in the pipeline. In the meantime, you can find some examples on the web on how to do that yourself using e.g. transposed convolutions. It is unfortunately much less efficient that per-channel upsampling, but correct is better than fast.

EDIT

They finally fixed this bug in TF 2.0:

image.resize now considers proper pixel centers (...).

This fix currently does not pass gradient through, which is... a bug that will hopefully also get fixed.

edited Jun 20 '20 at 09:12

Community

1
1

answered Sep 19 '18 at 07:47

P-Gn

23,115
9
87
104

Seems it produce same results as opencv resize also with `tensorflow=1.14.0` with `tensorflow.compat.v2` and `antialias=False` https://github.com/tensorflow/tensorflow/issues/6720#issuecomment-520638909 – mrgloom Sep 06 '19 at 11:33

Milind Deore · Answer 3 · 2018-12-20T14:59:01.237

Keras supports 'nearest' and 'bilinear' interpolation now with tensorflow backend. Check the documentation. But the default size value is indeed (2,2) or int value, so in that case your upsampling will be at least double.

This code needs Keras 2.2.3 or above. If you are using tf.keras, unfortunately, you cannot install keras separately, it is bundled as part of tensorflow installation. Hence either find the tf version or install the nightly build.

pip3 install --upgrade tf-nightly

Bilinear upsample in tensorflow?

3 Answers3