1

Hy guys

I need to implement Aitchison loss function for validation between input and output datasets. Data is 2 dimensional ( batch, features ). Batch dim constructed as 'None' size dimension for a while

If loss function could work with numpy arrays, it could be done easily approx in such way

def loss_Aitch(yTrue, yPred):
    yTrue_np = yTrue.numpy()
    yPred_np = yPred.numpy()

    sample_dist_mean = 0

    for i in range(yTrue_np.shape[0]):
        mult1 = 1.
        mult2 = 1.
        for j in range(yTrue_np.shape[1]):
            mult1 *= yTrue_np[i, j]
            mult2 *= yPred_np[i, j]

        mult1 = np.sqrt(mult1)
        mult2 = np.sqrt(mult2)

        sample_dist = 0
        for j in range(yTrue_np.shape[1]):
            sample_dist += np.square( np.log( yTrue_np[i, j] / mult1) - np.log(yPred_np[i, j] / mult2 ) )

        sample_dist = np.sqrt(sample_dist)
        sample_dist_mean += sample_dist

    sample_dist_mean /= yTrue_np.shape[0]

    return sample_dist_mean

but since tensors are placeholders, that's doesn't work.

So how this function could be directly implemented on tensors?

Anton
  • 73
  • 8

2 Answers2

0

You can simply accomplish this by replacing all NumPy functions to TensorFlow built-in functions. Here is the code:

def loss_Aitch(yTrue, yPred):
    sample_dist_mean = 0.

    for i in range(yTrue.shape[0]):
        mult1 = 1.
        mult2 = 1.
        for j in range(yTrue.shape[1]):
            mult1 *= yTrue[i, j]
            mult2 *= yPred[i, j]

        mult1 = tf.sqrt(mult1)
        mult2 = tf.sqrt(mult2)

        sample_dist = 0.
        for j in range(yTrue.shape[1]):
            sample_dist += tf.square(tf.math.log(yTrue[i, j] / mult1) - tf.math.log(yPred[i, j] / mult2))

        sample_dist = tf.sqrt(sample_dist)
        sample_dist_mean += sample_dist
        # print(sample_dist)
    sample_dist_mean /= yTrue.shape[0]

    return sample_dist_mean

and you can use this function like any other TensorFlow built-in loss:

model = Sequential([
    layers.Dense(64, activation="relu", input_shape=(32,)),
    layers.Dense(128, activation="relu"),
    layers.Dense(5),
])

model.compile(optimizer="adam", loss=loss_Aitch)

and then you can fit() with your data.

However, I noticed that this loss function may potentially cause nan when mult1 and mult2 are 0. And it may cause inf when they are extremely large, but since I am not familiar with the loss itself and I don't know what your data is like, I am also not sure what you want to do with this situation. A potential solution is to add epsilon to the denominator when dividing and clip by a value when the result gets too large.

seermer
  • 571
  • 6
  • 12
  • Sorry, but I guess that yTrue[i, j] on tensors is not correct. Moreover it can't work since that are placeholders and first dim just None dim – Anton Aug 03 '21 at 13:58
  • @Anton TensorFlow will infer batch size at runtime when you fit, at least I tried and it ran with no problem, and I am not sure what you mean by "yTrue[i, j] on tensors is not correct", because I have tried and TensorFlow has no problem accessing it – seermer Aug 03 '21 at 14:03
  • Okay, in general probably your approach works. But my own case is that I adding loss through "add_loss" function (it more convinient for many internal reasons than adding through "compile") , and in such way I guess you can't use tensor indexing – Anton Aug 04 '21 at 05:57
0

it seems that solution was found with regarding to answer of calculating geomean in tensorflow

geometric mean while calculationg tensorflow loss

So, my final solution is

def loss_A(y1, y2):
    logY1 = tf.math.log(y1)
    logY2 = tf.math.log(y2)

    log_G1 = tf.reduce_mean(logY1, axis = 1 )
    log_G2 = tf.reduce_mean(logY2, axis = 1 )

    log_G1 = tf.expand_dims(log_G1, 1)
    log_G2 = tf.expand_dims(log_G2, 1)


    logY1_sub = logY1 - log_G1
    logY2_sub = logY2 - log_G2


    A_v = tf.math.sqrt ( tf.reduce_sum( tf.math.square( logY1_sub - logY2_sub ), axis = 1 ) )


    return tf.reduce_mean(A_v)
Anton
  • 73
  • 8