Tensorflow: Dynamically Splitting Images into Pieces

Question

Im new to tensorflow and need help in managing my image dataset. Since the image set is huge the reading and training phases should overlap. Each image has also way more pixels in its width than in its height. I need to split the wide image into square parts. Since the width of the image varies, also the amount of parts will change.

It follows my example code, where I need help in getting it to work:

def main(unused_argv):
   filenames = tf.constant(['im_01.png', 'im_02.png', 'im_03.png', 'im_04.png'])
   labels = tf.constant([0, 1, 0, 1])

   def parse_fn(file, label):
      image = tf.image.decode_png(tf.read_file(file), channels=1)

       shape_list = image.get_shape().as_list()
       image_width = shape_list[0]
       image_height = shape_list[1]
       num_pieces = int(image_width/image_height)

       Xy_pairs = []
       for i in range(num_pieces):
          offset_width = i * image_height
          sub_image = tf.image.crop_to_bounding_box(image, 0, offset_width, image_height, image_height)
          sub_image = tf.image.resize_images(sub_image, [128, 128])
          Xy_pairs.append((sub_image, label))

       return Dataset.from_tensor_slices(Xy_pairs)


dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))
dataset = dataset.flat_map(parse_fn)

sess = tf.InteractiveSession()
it = dataset.make_one_shot_iterator()
while True:
    print('OUT: ' + it.get_next().eval())

The error looks like this TypeError: unsupported operand type(s) for /: 'NoneType' and 'NoneType' since tensorflow does not know the size of the images while computing the dataflow graph. Im pretty sure I need something like a placeholder for image.get_shape(). I hope the community can help me with this.

I wonder if you need to use `tf.shape` instead. https://stackoverflow.com/questions/37096225/how-to-understand-static-shape-and-dynamic-shape-in-tensorflow?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa — geometrikal, May 04 '18 at 14:02
Sound good, but `tf.shape(image)[0]` returns me a tensor. So how to dynamically extract the value out of it. Using eval() does not help me since it is used while defining the dataflow graph. I mean it throughs another error then there :/ — Harald, May 04 '18 at 14:13

benjaminplanche · Accepted Answer · 2018-05-04T15:14:25.723

As pointed out in comments, your image shape is left undefined in the graph, so trying to get the values before evaluation (through image.get_shape().as_list()) returns you (None, None, None).

To have everything dynamic in the graph, you could use tf.while_loop() instead of your current for loop to extract the patches, as shown below (this code may need some tweaking):

import tensorflow as tf

filenames = tf.constant(['im_01.png', 'im_02.png', 'im_03.png', 'im_04.png'])
labels = tf.constant([0, 1, 0, 1])
patch_size = [128, 128]

def parse_fn(file, label):
    image = tf.image.decode_png(tf.read_file(file), channels=1)

    shape= tf.shape(image)
    image_width = shape[1]
    image_height = shape[0]
    num_pieces = tf.div(image_width, image_height)

    def tf_while_condition(index, outputs):
        # We loop over the number of pieces:
        return tf.less(index, num_pieces)

    def tf_while_body(index, outputs):
        # We get the image patch for the given index:
        offset_width = index * image_height
        sub_image = tf.image.crop_to_bounding_box(image, 0, offset_width, image_height, image_height)
        sub_image = tf.image.resize_images(sub_image, patch_size)
        sub_image = tf.expand_dims(sub_image, 0)
        # We add it to the output patches (may be a cleaner way to do so):
        outputs = tf.concat([outputs[:index], sub_image, outputs[index + 1:]], axis=0)
        # We increment our index and return the values:
        index = tf.add(index, 1)
        return index, outputs

    # We build our patches tensor which will be filled with the loop:
    patches = tf.zeros((num_pieces, *patch_size, shape[2]), dtype=tf.float32)
    _, patches = tf.while_loop(tf_while_condition, tf_while_body, [0, patches])

    # We tile the label to have one per patch:
    patches_labels = tf.tile(tf.expand_dims(label, 0), [num_pieces])

    return tf.data.Dataset.from_tensor_slices((patches, patches_labels))

dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))
dataset = dataset.flat_map(parse_fn)

sess = tf.InteractiveSession()
it = dataset.make_one_shot_iterator()
op = it.get_next()
while True:
    res = sess.run(op)
    print(res)

This is way more complicated then i thought it would get, however great answer. Still there is somewhere a bug. It looks like the iterator after initializing the session does not contain any elements. `OutOfRangeError: [[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[128,128,?], []], output_types=[DT_FLOAT, DT_INT32]` — Harald, May 04 '18 at 14:59
It is just a matter of converting your loop in TF format, which is quite alright since you already got all the operations defined. :) As for the error, I just noticed a typo in your code: `image_width` is defined by `shape[1]` and `image_height` by `shape[0]`, not the opposite. Because of that, `num_pieces` was equal to `0`, hence the empty output. I corrected my own answer accordingly. — benjaminplanche, May 04 '18 at 15:17

Tensorflow: Dynamically Splitting Images into Pieces

1 Answers1