0

Problem definition:

I am implementing a CNN using Tensorflow. The Input and output are of size samples x 128 x 128 x 1 (grayscale image). In loss function I already have SSIM (0-1) and now my goal is to combine SSIM value with perceptual loss using pre-trained VGG16. I have already consulted following answers link1, link2 but instead of concatenating VGG model at end of the main model I would like to compute feature maps inside loss function at specific layers (e.g. pool1, pool2, pool3) and compute overall MSE. I have defined loss function as following:

Combined Loss function:

def lossfun( yTrue, yPred):
    alpha = 0.5
    return (1-alpha)*perceptual_loss(yTrue, yPred) + alpha*K.mean(1-tf.image.ssim(yTrue, yPred, 1.0))

and perceptual loss:

from tensorflow.keras.applications.vgg16 import VGG16
from tensorflow.keras.applications.vgg16 import preprocess_input
model = VGG16()
model = Model(inputs=model.inputs, outputs=model.layers[1].output)

def perceptual_loss(yTrue, yPred): 
    true = model(preprocess_input(yTrue))
    P=Concatenate()([yPred,yPred,yPred])
    pred = model(preprocess_input(P))
    vggLoss = tf.math.reduce_mean(tf.math.square(true - pred))
    return vggLoss

The Error I am running into is followig:

ValueError: Dimensions must be equal, but are 224 and 128 for 'loss_22/conv2d_132_loss/sub' (op: 'Sub') with input shapes: [?,224,224,64], [?,128,128,64].

Error arises due to following reason:

yPred has size None,128,128,1 , after concatenating it three time and pred = model(preprocess_input(P)) I receive feature map named pred of size None,128,128,64. While yTrue has size None and after true = model(preprocess_input(yTrue)) dimension of true is None,224,224,64. This eventually creates dimension incompatibility while computing final vggLoss.

Question

Since I am new to this task I am not sure if I am approaching the problem in the right manner. Should I create samples of size 224x244 instead of 128x128 in order to avoid this conflict, or is there any other workaround to fix this issue?

Thank you !

jklm
  • 171
  • 11

0 Answers0