7

I used the first example here as an example of network.

How to stop the training when the loss reach a fixed value ?

So, for example, I would like to fix a maximum of 3000 epochs and the training will stop when the loss will be under 0.2.

I read this topic but it is not the solution I found.

I would want to stop the training when the loss reach a value, not when there is no improvement like with this function proposed in the precedent topic.

Here is the code:

import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.optimizers import SGD

# Generate dummy data
import numpy as np
x_train = np.random.random((1000, 20))
y_train = keras.utils.to_categorical(np.random.randint(10, size=(1000, 1)), num_classes=10)
x_test = np.random.random((100, 20))
y_test = keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=10)

model = Sequential()
# Dense(64) is a fully-connected layer with 64 hidden units.
# in the first layer, you must specify the expected input data shape:
# here, 20-dimensional vectors.
model.add(Dense(64, activation='relu', input_dim=20))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',
              optimizer=sgd,
              metrics=['accuracy'])

model.fit(x_train, y_train,
          epochs=3000,
          batch_size=128)
score = model.evaluate(x_test, y_test, batch_size=128)
Julien
  • 763
  • 8
  • 32
  • Did you read all the answers there? To quote one of the answers, The keras.callbacks.EarlyStopping callback does have a min_delta argument. From Keras documentation: min_delta: minimum change in the monitored quantity to qualify as an improvement, i.e. an absolute change of less than min_delta, will count as no improvement. – Dinesh Jul 20 '18 at 10:13
  • Possible duplicate of [How to tell Keras stop training based on loss value?](https://stackoverflow.com/questions/37293642/how-to-tell-keras-stop-training-based-on-loss-value) – Dinesh Jul 20 '18 at 10:14
  • It is not what I looked for. I would want to stop the training when the loss reach a fixed value. I edited my question – Julien Jul 20 '18 at 10:25
  • @Julien The most-voted answer to the question you referred to is **exactly** what you are looking for. When the loss reaches below a specified value it stops training. – today Jul 20 '18 at 20:50
  • For TF2, have a look at using a Custom Callback. https://www.tensorflow.org/guide/keras/custom_callback#early_stopping_at_minimum_loss – Ben Butterworth Apr 04 '21 at 12:38

3 Answers3

7

You can use some method like this if you would switch to TensorFlow 2.0:

class haltCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs={}):
    if(logs.get('loss') <= 0.05):
        print("\n\n\nReached 0.05 loss value so cancelling training!\n\n\n")
        self.model.stop_training = True

You just need to create a callback like that and then add that callback to your model.fit so it becomes something like this:

model.fit(x_train, y_train,
      epochs=3000,
      batch_size=128,
      callbacks=['trainingStopCallback'])

This way, the fitting should stop whenever it reaches down below 0.05 (or whatever value you put on while defining it).

Also, since it's been a long time you asked this question but it still has no actual answer for using it with TensorFlow 2.0, I updated your code snippet to TensorFlow 2.0 so everyone can now easily find and use it with their new projects.

import tensorflow as tf

# Generate dummy data
import numpy as np


x_train = np.random.random((1000, 20))
y_train = tf.keras.utils.to_categorical(
    np.random.randint(10, size=(1000, 1)), num_classes=10)
x_test = np.random.random((100, 20))
y_test = tf.keras.utils.to_categorical(
    np.random.randint(10, size=(100, 1)), num_classes=10)

model = tf.keras.models.Sequential()
# Dense(64) is a fully-connected layer with 64 hidden units.
# in the first layer, you must specify the expected input data shape:
# here, 20-dimensional vectors.
model.add(tf.keras.layers.Dense(64, activation='relu', input_dim=20))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dropout(0.5))
model.add(tf.keras.layers.Dense(10, activation='softmax'))


class haltCallback(tf.keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs={}):
        if(logs.get('loss') <= 0.05):
            print("\n\n\nReached 0.05 loss value so cancelling training!\n\n\n")
            self.model.stop_training = True


trainingStopCallback = haltCallback()

sgd = tf.keras.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',
              optimizer=sgd,
              metrics=['accuracy', 'loss'])

model.fit(x_train, y_train,
          epochs=3000,
          batch_size=128,
          callbacks=['trainingStopCallback'])
score = model.evaluate(x_test, y_test, batch_size=128)
msteknoadam
  • 81
  • 1
  • 4
0

Documentation here : EarlyStopping 

keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0, patience=0, verbose=0, mode='auto', baseline=None)
Julien
  • 763
  • 8
  • 32
VegardKT
  • 1,226
  • 10
  • 21
  • Thank you for your answer, but it is not what I looked for... I would want to stop the training when the loss reach a fixed value. – Julien Jul 20 '18 at 13:10
  • Sorry Julien, you're right. I'm not able to find what you want in Keras, but as Dinesh commented, have a look at this. That should do the trick. https://stackoverflow.com/questions/37293642/how-to-tell-keras-stop-training-based-on-loss-value – VegardKT Jul 23 '18 at 06:42
0

I solved it by doing this:

history = model.fit(training1, training2, epochs=100, verbose=True)

loss=history.history["loss"]

for x in loss: 

    if x <= 1:

        print("Reached 1 loss value, cancelling training")
        model.stop_training = True 

        break
buddemat
  • 4,552
  • 14
  • 29
  • 49
ajooo
  • 1
  • 1