How to assign an image to an image patch

Question

I am trying to implement a cut-out strategy as a data augmentation method. For this strategy, a random rectangular region of an image is selected and replaced by random noise. However, I could not figure out how to "assign" a random image (with shape HxWxD) to a rectangular region of a larger image. A MATLAB code to what I need would be something like:

large_image(patch_start_y:patch_end_y, patch_start_x:patch_end_x, :) = random_noise_image

Following is my efforts to achieve an equivalent result. This is how I get a random image:

random_patch = tf.random_uniform([cut_out_height, cut_out_width, tf.constant(3)], minval=tf.constant(0),
                                 maxval=tf.constant(256), dtype=tf.int32)

random_patch = tf.expand_dims(random_patch, 0)

And this is how I tried to assign it to an existing image, having a shape of (1, ?, ?, 3)

return tf.assign(image[0, cut_out_start_y:(cut_out_start_y+cut_out_height),
          cut_out_start_x:(cut_out_start_x+cut_out_width), :], random_patch, validate_shape=False)

However, I get the following error:

ValueError: Shapes must be equal rank, but are 1 and 0
    From merging shape 2 with other shapes. for 'strided_slice/stack_2' (op: 'Pack') with input shapes: [], [1], [1], [].

Any ideas?

Update: I have come to the conclusion that the problem is the image and the patch being a tf.Tensor. For tf.Variable type, tf.assign works just fine. See below MWE for the problem:

import tensorflow as tf
image = [[1,2,3,4],[5,6,7,8], [9, 10, 11,12], [13,14,15,16]]
patch = [[9, 8],[7,6]]
image_tensor = tf.convert_to_tensor(image)
patch_tensor = tf.convert_to_tensor(patch)
modified_image = tf.assign(image_tensor[1:3, 1:3], patch_tensor)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(sess.run(modified_image))

It gives:

ValueError: Sliced assignment is only supported for variables

Is there a workaround to achieve what I want?

If first dimension in shape is 1, the first index should be 0, not 1. — Mad Physicist, Aug 22 '18 at 15:40
@MadPhysicist, you are right. However, I have changed it and I still get the same error. Image is a tf.Tensor. — talkanat, Aug 22 '18 at 15:42
I'm pretty sure you don't need an explicit `tf.constant`, even if the conversion will happen under the hood anyway. — Mad Physicist, Aug 22 '18 at 15:42
Does this post work? https://stackoverflow.com/q/39157723/2988730 — Mad Physicist, Aug 22 '18 at 15:43
`return image[0, cut_out_start_y:cut_out_start_y+cut_out_height, cut_out_start_x:cut_out_start_x+cut_out_width, :].assign(random_patch)` — Mad Physicist, Aug 22 '18 at 15:47
Sorry, one more step: `random_patch = tf.random_uniform([tf.constant(1), cut_out_height, cut_out_width, tf.constant(3)], minval=tf.constant(0), maxval=tf.constant(256), dtype=tf.int32)` gotta make those shapes match — Mad Physicist, Aug 22 '18 at 15:53
@MadPhysicist, I did not include in the code sample above but they are already of the same size. I ensured that by using tf.expand_dims on patch. But for clarity I will add this to code also. — talkanat, Aug 22 '18 at 15:57

How to assign an image to an image patch

0 Answers0