How do I change the last layer in Keras to get logits rather than probabilities from model?

Question

My goal is to retrieve logits from a Keras neural network model. I read here: Keras - how to get unnormalized logits instead of probabilities

That I need to change the last activation layer to "linear." Here's my code

from __future__ import absolute_import, division, print_function

import tensorflow as tf
from tensorflow import keras

import numpy as np
import matplotlib.pyplot as plt

#Preprocessing
fashion_mnist = keras.datasets.fashion_mnist

(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 
               'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
train_images = train_images / 255.0
test_images = test_images / 255.0
#Preprocessing

#Generate the Model
model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=tf.nn.linear)
])

model.compile(optimizer='adam', 
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
model.fit(train_images, train_labels, epochs=5)

The problem happens on this line. Is there a better way to get the logits? If not how do I get the activation to be linear?

keras.layers.Dense(10, activation=tf.nn.linear)

Do you have a trained model and want to get logits without retraining, or you want to have a model that produces logits and then train it? You mention a problem but there are no details. — Dr. Snoopy, Jun 13 '19 at 20:37
@MatiasValdenegro I need the former, but I didn't know of a way of doing it, so I basically implemented the latter. In the line of code with "keras.Sequential" the last layer is linear, which produces logits right? — JobHunter69, Jun 13 '19 at 22:12

score 0 · Accepted Answer · answered Jun 14 '19 at 01:22

You can use Activation layer

#Generate the Model
model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation=tf.nn.relu),
    keras.layers.Dense(10, activation=None),
    keras.layers.Activation('relu')
])

Manoj Mohan · Answer 2 · 2019-06-14T05:01:55.037

Using linear activation is correct for getting the logits. As you're developing keras models, use the activations from keras.

Replace

keras.layers.Dense(128, activation=tf.nn.relu),
keras.layers.Dense(10, activation=tf.nn.linear)

with

keras.layers.Dense(128, activation=keras.activations.relu),  
keras.layers.Dense(10, activation=keras.activations.linear)

Alternately, not specifying an activation also defaults to linear.

keras.layers.Dense(10)

How do I change the last layer in Keras to get logits rather than probabilities from model?

2 Answers2

Linked