Implementing a custom loss function in Keras with a term like l1 regularisation

Question

I am creating my own loss function (that I want to use in eager execution in Keras). I would like to add to it a term similar to a l1 loss function.

That is the loss function I am using now

def loss(model, x, y, x_dev, y_dev, variables):
  y_ = model(x)
  y_dev_ = model(x_dev)

  y_temp = 1.5

  return loss_mae(y_true=y, y_pred=y_)+y_temp*
                K.mean(tf.convert_to_tensor(variables))

with

import keras.backend as K
def loss_mae(y_true, y_pred):
    return K.mean(K.abs(y_pred-y_true))

my idea is to add to my loss function a constant (y_temp) and then I would like to multiply it for the trainable variables (to achieve something similar to a l1 regularisation term).

I tried passing to the loss() function model.trainable_variables but that does not work and I get a

TypeError: can't multiply sequence by non-int of type 'numpy.float32'

anyone has any suggestions?

Answer 1

The reason you are getting this error is because in Python your expression y_temp*variables means something like "take variables y_temp times and merge them in one sequence".

In other words, 2 * [1, 2, 3] is not [4, 5, 6], but [1, 2, 3, 1, 2, 3]. Obviously, it makes little sense to use anything but integer in this case.

If I understand you correctly, you are trying to perform element-wise multiplication. To achieve this you should use list comprehension, something like [x * 1.5 for x in [1, 2, 3]]

By the way, you could also check these answers to similar questions: Python can't multiply sequence by non-int of type 'float' and How do I multiply each element in a list by a number?