Custom loss and accuracy in Keras

Question

I want to extend the output of the network to two outputs in this code

# Model architecture
  input = Input(shape = (max_len,))
  model = Embedding(input_dim = len(words) + 2, output_dim = embedding, input_length = max_len, mask_zero = True)(input)
  model = Bidirectional(LSTM(units = 50, return_sequences=True, recurrent_dropout=0.1))(model)
  model = TimeDistributed(Dense(50, activation="relu"))(model)
  crf = CRF(num_tags+1)  # CRF layer
  out = crf(model)  # output

  model = Model(input, out)
  model.compile(optimizer="rmsprop", loss=crf.loss_function, metrics=[crf.accuracy])

(The original code is at https://github.com/Akshayc1/named-entity-recognition.git). I want the loss function to be the summation of losses for these two outputs, and accuracy to be mean of accuracy for these two outputs, I followed the available instructions:

model = TimeDistributed(Dense(50, activation="relu"))(model)
model1 = TimeDistributed(Dense(50, activation="relu"))(model)
crf1 = CRF(num_tags+1)  # CRF layer
crf2 = CRF(num_tags+1)
out1 = crf1(model)  # output
out2 = crf2(model1) 

model = Model(input, [out1, out2])

def custom_loss(y_true, y_pred1,  y_pred2):
    loss1 = losses.categorical_crossentropy(y_true,y_pred1)
    loss2 = losses.categorical_crossentropy(y_true,y_pred2)
    return (loss1 + loss2)/2


def Custom_accuracy(y_true, y_pred1,y_pred2, k=10):
    acc1 = K.mean(K.in_top_k(y_pred, K.argmax(y_true, axis=-1), k), axis=-1)
    acc2 = K.mean(K.in_top_k(y_pred1, K.argmax(y_true, axis=-1), k), axis=-1)
    acc = (acc1 + acc2)/2
    return acc
model.compile(optimizer="rmsprop", loss=custom_loss,metrics=Custom_accuracy)

but this shows the error:

TypeError: custom_loss() missing 1 required positional argument: 'y_pred2'

Please see https://stackoverflow.com/help/minimal-reproducible-example to help you create a minimal, reproducible example. (While the github code is helpful, not all of us can access this easily.) — rajah9, Mar 18 '20 at 13:55
Does this answer your question? [Make a custom loss function in keras](https://stackoverflow.com/questions/45961428/make-a-custom-loss-function-in-keras) — Vlad, Mar 18 '20 at 14:11
@Vlad, thanks for the link. I followed it and created a custom loss and accuracy, but the problem is how I can feed two outputs for the loss and accuracy? now for the loss I am defining `def custom_loss(y_true, y_pred1, y_pred2):` but the error is `TypeError: custom_loss() missing 1 required positional argument: 'y_pred2'` — Niki, Mar 18 '20 at 18:28

rami · Answer 1 · 2020-03-22T00:14:16.213

1

It can be solved by passing two loss functions to loss argument in model.compile than to pass three variables in loss function as described in the documentation and also make classes for custom metric and loss. Make the following changes -

...

crf1 = CRF(num_tags+1,name="out1") <-- # change 1
crf2 = CRF(num_tags+1,name="out2") <-- # change 2
out1 = crf1(model)  
out2 = crf2(model1) 

model = Model(input, [out1, out2])

<define accuracy class and create its object> <-- # change 3
<define loss class and create its object> <-- # change 4

model.compile(optimizer="rmsprop", 
            loss={"out1":<loss_object_1>,"out2":<loss_object_2>},
            metrics={"out1":<accuracy_object_1>,"out2":<accuracy_object_2>}) <-- # change 5

edited Mar 22 '20 at 00:14

answered Mar 20 '20 at 03:36

rami

211
2
6

thanks. in `out1 = crf1(model,name="out1") ` it gives this error: `TypeError: call() got an unexpected keyword argument 'name'` – Niki Mar 21 '20 at 12:53
My bad! I passed ```name``` to class call function instead of class definition(or initialization). I have updated the answer. – rami Mar 22 '20 at 00:12
thanks, @rami following your approach that problem is solved, but I have to change my custom function from `def Custom_loss(self, y_true, y_pred1, y_pred2):' to ` def Custom_loss(self, y_true, y_pred):' meaning just considering one output, now I wonder how the averaging is done over these two losses? I think right now the final loss is equal to loss=loss1+loss2 but I'm looking for loss=(loss1+loss2)/2 – Niki Apr 29 '20 at 11:57

score 0 · Answer 2 · answered Mar 18 '20 at 19:29

0

I think that is because the output of the model is not two outputs values( like o1,o2), but a list of two values(like [o1,o2]). Try this custom loss function, hopefully it gets the issue gone:

def custom_loss(y_true, y_predsList):
    y_pred1,  y_pred2 = y_predsList
    loss1 = losses.categorical_crossentropy(y_true,y_pred1)
    loss2 = losses.categorical_crossentropy(y_true,y_pred2)
    return (loss1 + loss2)/2

answered Mar 18 '20 at 19:29

alift

1,855
2
13
28

thanks. I tried that but in custom_loss `y_pred1, y_pred2 = y_predsList` it gives this error `TypeError: Tensor objects are only iterable when eager execution is enabled. To iterate over this tensor use tf.map_fn.` then I tried `import tensorflow as tf , tf.enable_eager_execution()` but this gives error `RuntimeError: tf.placeholder() is not compatible with eager execution.` – Niki Mar 19 '20 at 12:13

Custom loss and accuracy in Keras

2 Answers2