How to run keras model twice before loss?

Question

I want to compute a loss function which uses output of the network twice on different inputs. For example as hypothetically,

first_output = model(first_input)
second_output = model(second_input)
loss = mean_absolute_error(first_output, second_output)

How to achieve this in tensorflow or keras?

Update: Thank you guys for replies. I want to reimplement this paper in keras or tensorflow. As explained in it, "critic" network which is discriminator in GAN has two inputs and run through them one by one and compute loss function depending on outputs and compute gradient. Main problem is how to make possible in tensorflow or keras?

[How to concatenate two layers](https://stackoverflow.com/questions/43196636/how-to-concatenate-two-layers-in-keras) may be helpful. — mickey, Jan 28 '19 at 15:57
Here is [the paper author's GitHub repository implementing SegAN with PyTorch](https://github.com/YuanXue1993/SegAN). I don't think you will have an easy time getting somebody on this site to translate that to Keras/TF for you, but we can definitely help with specific questions you come across! — Engineero, Jan 28 '19 at 21:11
What problem are you having? Can you show us a concrete example that is not giving you what you want, as well as describe what you expect to get? — Engineero, Jan 28 '19 at 21:14
Sorry for my question being too general. Problematic part for me is that, in the original implementation, author trained the "critic" network once with input image masked with output from "segmentor" network and then again with input image masked with ground before computing the loss. So basically, he run the two separate inputs through the same model before getting the loss. As I understand, in pytorch, it is possible as the pytorch is dynamic. But how to do this with tensorflow? My current guess is to share layers of the critic between two models for two inputs and have the single loss? — Odgiiv, Jan 28 '19 at 21:26

Engineero · Answer 1 · 2019-01-28T16:08:56.470

You could try using keras.layers.merge. I have used this before to make Siamese networks with something like:

first_output = model(first_input)
second_output = model(second_input)
mae = lambda x: mean_absolute_error(x[0], x[1])
distance = merge(inputs=[first_output, second_output],
                 mode=mae,
                 output_shape=lambda x: x[0],
                 name='mean_absolute_error')

For the Siamese network example, you'd then typically make some prediction on this distance measure with something like:

prediction = Dense(2, activation='softmax', name='prediction')(distance)
model = Model([first_input, second_input], prediction, name='siamese_net')
model.compile(optimizer=Adam(),
              loss=some_loss_function)

using keras.models.Model and keras.layers.Dense for that example.

Note that keras.layers.merge is (I believe) deprecated in the latest versions of Keras, which is really a shame. I think to do something similar with the most modern Keras you would need to use keras.layers.Concatenate to combine the two results followed by keras.layers.Lambda to apply the function.

Wouldn't predicting zeros for all inputs be optimal for such a model? — Mikhail Berlinkov, Jan 28 '19 at 16:08
@MikhailBerlinkov Depends what `some_loss_function` is and what your labels look like. In the example shown, and in most Siamese networks, you're outputting whether the network thinks two inputs are the same or different. If your `model` always output zeros, presumably you would always predict the same class and you'd be no better than baseline for your dataset. So I would say not optimal. — Engineero, Jan 28 '19 at 16:13

How to run keras model twice before loss?

1 Answers1