How can I determine the "index" of y_true and y_pred in the loss function in keras?

Question

I have to design a network in which the direction of the eigenvectors should be predicted as close as possible to the groundtruth.I'm working with heart MRI-Data and I already have a function which computes the eigenvector of every voxel. My problem is, that the function needs more inputs than just y_pred and y_true to compute the eigenvectors. My y_true/y_pred has dimensions (841,336,336,33) (meaning of every dimension:(index,x-direction,y-direction,channel)). I now need to know the index at any given time, so that I can pass the right index to my additional data. Is there any way to know which image (of the 841) the loss function is currently looking at?

Answer 1

You might be best served by writing a custom loss function that takes into account your additional data and eigenvector calculation function. In particular, I believe you should be able to pass additional information (at the same index) within the "y_true" array, and then slice into it as needed (using tensorflow functionality to separate out the various components.) Here is an example that shows the idea. Please note that the top section just allows for reproducible results on Google Colab (CPU). The main code is after the comment:"# Rest of code follows ..." I hope this helps.

# Install TensorFlow
try:
  # %tensorflow_version only exists in Colab.
  %tensorflow_version 2.x
except Exception:
  pass

import tensorflow as tf
print(tf.__version__)
print(tf.executing_eagerly())

# Setup repro section from Keras FAQ with TF1 to TF2 adjustments

import numpy as np
import random as rn

# The below is necessary for starting Numpy generated random numbers
# in a well-defined initial state.

np.random.seed(42)

# The below is necessary for starting core Python generated random numbers
# in a well-defined state.

rn.seed(12345)

# Force TensorFlow to use single thread.
# Multiple threads are a potential source of non-reproducible results.
# For further details, see: https://stackoverflow.com/questions/42022950/

session_conf = tf.compat.v1.ConfigProto(intra_op_parallelism_threads=1,
                                        inter_op_parallelism_threads=1)

# The below tf.set_random_seed() will make random number generation
# in the TensorFlow backend have a well-defined initial state.
# For further details, see:
# https://www.tensorflow.org/api_docs/python/tf/set_random_seed

tf.compat.v1.set_random_seed(1234)

sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=session_conf)
tf.compat.v1.keras.backend.set_session(sess)

# Rest of code follows ...

# Custom Loss
def my_custom_loss(y_true, y_pred):

    tf.print('inside my_custom_loss:')
    tf.print('y_true:')
    tf.print(y_true)
    tf.print('y_true column 0:')
    tf.print(y_true[:,0])
    tf.print('y_true column 1:')
    tf.print(y_true[:,1])
    tf.print('y_pred:')
    tf.print(y_pred)

    y_zeros = tf.zeros_like(y_pred)
    y_mask = tf.math.greater(y_pred, y_zeros)
    res = tf.boolean_mask(y_pred, y_mask)
    logres = tf.math.log(res)
    finres = tf.math.reduce_sum(logres)

    return finres

# Define model
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(1, activation='linear', input_dim=1, name="Dense1"))
model.compile(optimizer='rmsprop', loss=my_custom_loss)
print('model.summary():')
print(model.summary())

# Generate dummy data
data = np.array([[2.0],[1.0],[1.0],[3.0],[4.0]])
labels = np.array([[[2.0],[1.0]],
                   [[0.0],[3.0]],
                   [[0.0],[3.0]],
                   [[0.0],[3.0]],
                   [[0.0],[3.0]]])

# Train the model.
print('training the model:')
print('-----')
model.fit(data, labels, epochs=1, batch_size=5)
print('done training the model.')

print(data.shape)
print(labels.shape)

a = model.predict(data)
print(a)