Keras custom loss function with Mahalanobis distance loss how to

Question

I am trying to implement a custom loss function in Keras using Mahalanobis distance loss. however I always run into this annoying ERROR.

Mahalanobis distance (or "generalized squared interpoint distance" for its squared value[3]) can also be defined as a dissimilarity measure between two random vectors x and y of the same distribution with the covariance matrix S.

d(x,y) = square [Transpose(x-y) * Inverse(S)* (x-y)]

(https://en.wikipedia.org/wiki/Mahalanobis_distance)

n_classes = 4
n_samples=800
X, y = make_classification(n_samples=n_samples, n_features=20, n_informative=4, n_redundant=0, n_classes=n_classes, n_clusters_per_class=2)
y = to_categorical(y)
Xtrainb, testXb, ytrainb, ytestb = train_test_split(X, y, test_size = 0.3, random_state=42)

x_trainb = np.reshape(Xtrainb, (Xtrainb.shape[0], Xtrainb.shape[1], 1))
Xtestb = np.reshape(testXb, (testXb.shape[0], testXb.shape[1], 1))

densesize = 4
input_datab = Input(shape=(Xtrainb.shape[1],1)) 
epochs = 10
batch_size = 32
dropout= 0.1
lr= 0.001

########
def mahalanobis(y_true, y_pred):
    x_minus_mn_with_transpose = K.transpose(y_true - y_pred)
    Covariance = covr1(y_true, y_pred)
    inv_covmat = tf.linalg.inv(Covariance)
    x_minus_mn = y_true - y_pred
    left_term = K.dot(x_minus_mn, inv_covmat)
    D_square = K.dot(left_term, x_minus_mn_with_transpose)
    return D_square 

def covr1(y_true, y_pred):
    #x_mean = K.mean(y_true)
    #y_mean = K.mean(y_pred)
    Cov_numerator = K.sum(((y_true - y_pred)*(y_true - y_pred)))
    Cov_denomerator = len(Xtrainb)-1
    Covariance = (Cov_numerator / Cov_denomerator)
    return Covariance


conv1= Conv1D(filters=80, kernel_size=2, padding='same',   input_dim=Xtrainb.shape[1])(input_datab)
maxpool = MaxPooling1D(pool_size=3, stride=3 )(conv1)
conv2= Conv1D(filters=50, kernel_size=2, padding='same',   input_dim=Xtrainb.shape[1])(maxpool)
maxpool = MaxPooling1D(pool_size=3, stride=3)(conv2)
flatten = Flatten()(maxpool)
dense = Dense(84, activation='relu')(flatten)
dense = Dense(1024, activation='relu')(flatten)
dense = Dense(densesize, activation='softmax')(dense)
model = Model(inputs=[input_datab],outputs=[dense])
model.compile(loss= mahalanobis,  optimizer='adam', metrics=['acc'])
hist = model.fit(x_trainb, ytrainb, validation_data=(Xtestb, ytestb), epochs=epochs, batch_size=batch_size)

ValueError: Shape must be at least rank 2 but is rank 0 for 'loss_88/dense_270_loss/MatrixInverse' (op: 'MatrixInverse') with input shapes: [].

Did you ever manage to answer this question? – ABIM Mar 11 '20 at 10:30 — ABIM, Mar 11 '20 at 10:30

score 0 · Answer 1 · answered Oct 09 '20 at 17:07

the problem of your code is when computing the Covariance matrix

there is my Mahalanobis distance: Hope this work for you; because its works for me :)

def mahala_dist(m, n):
 diff = m - n
 cov = tfp.stats.covariance(tf.transpose(n))
 mull = K.dot(tf.linalg.inv(cov), diff)
 mull2 = K.dot(mull, tf.transpose(diff))
 dist = tf.sqrt(mull2)
 return dist

Its work with both TF and Keras frameworks. good luck.

Keras custom loss function with Mahalanobis distance loss how to

1 Answers1