Tensorflow read images with labels

Question

I am building a standard image classification model with Tensorflow. For this I have input images, each assigned with a label (number in {0,1}). The Data can hence be stored in a list using the following format:

/path/to/image_0 label_0
/path/to/image_1 label_1
/path/to/image_2 label_2
...

I want to use TensorFlow's queuing system to read my data and feed it to my model. Ignoring the labels, one can easily achieve this by using string_input_producer and wholeFileReader. Here the code:

def read_my_file_format(filename_queue):
  reader = tf.WholeFileReader()
  key, value = reader.read(filename_queue)
  example = tf.image.decode_png(value)
  return example

#removing label, obtaining list containing /path/to/image_x
image_list = [line[:-2] for line in image_label_list]

input_queue = tf.train.string_input_producer(image_list)                                                     
input_images = read_my_file_format(input_queue)

However, the labels are lost in that process as the image data is purposely shuffled as part of the input pipeline. What is the easiest way of pushing the labels together with the image data through the input queues?

Answer 1

Using slice_input_producer provides a solution which is much cleaner. Slice Input Producer allows us to create an Input Queue containing arbitrarily many separable values. This snippet of the question would look like this:

def read_labeled_image_list(image_list_file):
    """Reads a .txt file containing pathes and labeles
    Args:
       image_list_file: a .txt file with one /path/to/image per line
       label: optionally, if set label will be pasted after each line
    Returns:
       List with all filenames in file image_list_file
    """
    f = open(image_list_file, 'r')
    filenames = []
    labels = []
    for line in f:
        filename, label = line[:-1].split(' ')
        filenames.append(filename)
        labels.append(int(label))
    return filenames, labels

def read_images_from_disk(input_queue):
    """Consumes a single filename and label as a ' '-delimited string.
    Args:
      filename_and_label_tensor: A scalar string tensor.
    Returns:
      Two tensors: the decoded image, and the string label.
    """
    label = input_queue[1]
    file_contents = tf.read_file(input_queue[0])
    example = tf.image.decode_png(file_contents, channels=3)
    return example, label

# Reads pfathes of images together with their labels
image_list, label_list = read_labeled_image_list(filename)

images = ops.convert_to_tensor(image_list, dtype=dtypes.string)
labels = ops.convert_to_tensor(label_list, dtype=dtypes.int32)

# Makes an input queue
input_queue = tf.train.slice_input_producer([images, labels],
                                            num_epochs=num_epochs,
                                            shuffle=True)

image, label = read_images_from_disk(input_queue)

# Optional Preprocessing or Data Augmentation
# tf.image implements most of the standard image augmentation
image = preprocess_image(image)
label = preprocess_label(label)

# Optional Image and Label Batching
image_batch, label_batch = tf.train.batch([image, label],
                                          batch_size=batch_size)

See also the generic_input_producer from the TensorVision examples for full input-pipeline.

Answer 2

There are three main steps to solving this problem:

1. Populate the tf.train.string_input_producer() with a list of strings containing the original, space-delimited string containing the filename and the label.

2. Use tf.read_file(filename) rather than tf.WholeFileReader() to read your image files. tf.read_file() is a stateless op that consumes a single filename and produces a single string containing the contents of the file. It has the advantage that it's a pure function, so it's easy to associate data with the input and the output. For example, your read_my_file_format function would become:

   def read_my_file_format(filename_and_label_tensor):
     """Consumes a single filename and label as a ' '-delimited string.
   
     Args:
       filename_and_label_tensor: A scalar string tensor.
   
     Returns:
       Two tensors: the decoded image, and the string label.
     """
     filename, label = tf.decode_csv(filename_and_label_tensor, [[""], [""]], " ")
     file_contents = tf.read_file(filename)
     example = tf.image.decode_png(file_contents)
     return example, label
   

3. Invoke the new version of read_my_file_format by passing a single dequeued element from the input_queue:

   image, label = read_my_file_format(input_queue.dequeue())         
   

You can then use the image and label tensors in the remainder of your model.

Answer 3

In addition to the answers provided there are few other things you can do:

Encode your label into the filename. If you have N different categories you can rename your files to something like: 0_file001, 5_file002, N_file003. Afterwards when you read the data from a reader key, value = reader.read(filename_queue) your key/value are:

The output of Read will be a filename (key) and the contents of that file (value)

Then parse your filename, extract the label and convert it to int. This will require a little bit of preprocessing of the data.

Use TFRecords which will allow you to store the data and labels at the same file.