1

For semantic image segmentation, I understand that you often have a folder with your images and a folder with the corresponding masks. In my case, I have gray-scale images with the dimensions (32, 32, 32). The masks naturally have the same dimensions. The labels are saved as intensity values (value 1 = label 1, value 2 = label 2 etc.). 4 classes in total. Imagine I have found a model that was built with the keras model API. How do I know how to prepare my label data for it to be accepted by the model? Does it depend on the loss function? Is it defined in the model (Input parameter). Do I just add another dimension (4, 32, 32, 32) in which the 4 represents the 4 different classes and one-hot code it?

I want to build a 3D convolutional neural network for semantic segmentation but I fail to understand how to feed in the data correctly in keras. The predicted output is supposed to be a 4-channel 3D image, each channel showing the probability values of each pixel to belong to a certain class.

Johannes Schmidt
  • 371
  • 3
  • 12
  • So you are asking how to input image, with your own size for some pre trained model like VGG16, Inception, etc.? – Raguel Jul 04 '19 at 13:24
  • It doesn't have to be a pre trained model, just any model suitable for 3D grayscale images so that I can train the model. I just don't understand where it says in which format my label data has to be...keras in general – Johannes Schmidt Jul 04 '19 at 13:51
  • Well most of the times input layer is `0` layer. So doing `model.layers[0] = Input(shape(width, height, num_of_channels))` should work. It might be different for different models. (Not tested!) – Raguel Jul 04 '19 at 14:17
  • also found this https://stackoverflow.com/questions/42187425/how-to-change-input-shape-in-sequential-model-in-keras – Raguel Jul 04 '19 at 14:20
  • Here is an example: def isensee2017_model(input_shape=(4, 128, 128, 128), n_base_filters=16, depth=5, n_segmentation_levels=3, n_labels=4, optimizer=Adam, initial_learning_rate=5e-4, loss_function=weighted_dice_coefficient_loss, activation_name="sigmoid"): – Johannes Schmidt Jul 04 '19 at 15:22
  • Does that mean that my label_data has to be in the shape (4, 128, 128, 128)? What about the image_data? – Johannes Schmidt Jul 04 '19 at 15:24

1 Answers1

0

The Input() function defines the shape of the input tensor of a given model. For 3D images, often a 5D Tensor is expected, e.g. (None, 32, 32, 32, 1), where None refers to the batch size. Therefore the training images and labels have to be reshaped. Keras offers the to_categorical function to one-hot encode the label data (which is necessary). The use of generators helps to feed in the data. In this case, I cannot use the ImageDataGenerator from keras as it can only deal with RGB and grayscale images and therefore have to write a custom script.

Johannes Schmidt
  • 371
  • 3
  • 12