0

I am training a network using supervised learning for autonomous driving... The model is based off of the nvidia model. It seems to be training well with a loss of 0.044 and a validation loss of 0.033. Using the model in my environment the car drives well, so it is learning OK. But when I add the accuracy metric, the accuracy shows as 0.11 and never really changes.

Code is below. Am I not monitoring the accuracy correctly for regression? Or not understanding it?

def nvidia_model():
    model = Sequential()
    model.add(Convolution2D(24, 5, 5, subsample=(2, 2), input_shape=(66, 200, 3), activation='elu'))
    model.add(Convolution2D(36, 5, 5, subsample=(2, 2), activation='elu'))
    model.add(Convolution2D(48, 5, 5, subsample=(2, 2), activation='elu'))
    model.add(Convolution2D(64, 3, 3, activation='elu'))
    model.add(Convolution2D(64, 3, 3, activation='elu'))
#   model.add(Dropout(0.5))
    model.add(Flatten())
    model.add(Dense(100, activation = 'elu'))
#   model.add(Dropout(0.5))
    model.add(Dense(50, activation = 'elu'))
#   model.add(Dropout(0.5))
    model.add(Dense(10, activation = 'elu'))
#   model.add(Dropout(0.5))
    model.add(Dense(1))
    optimizer = Adam(lr=1e-3)
    model.compile(loss='mse', optimizer=optimizer, metrics=['accuracy'])
    return model

model = nvidia_model()
print(model.summary())
history = model.fit_generator(batch_generator(X_train, y_train, 100, 1),
                                  steps_per_epoch=300, 
                                  epochs=10,
                                  validation_data=batch_generator(X_valid, y_valid, 100, 0),
                                  validation_steps=200,
                                  verbose=1,
                                  shuffle = 1)
keineahnung2345
  • 2,635
  • 4
  • 13
  • 28
DaveS
  • 105
  • 1
  • 1
  • 8
  • 4
    Accuracy is meaningless in regression settings; please see [What function defines accuracy in Keras when the loss is mean squared error (MSE)?](https://stackoverflow.com/questions/48775305/what-function-defines-accuracy-in-keras-when-the-loss-is-mean-squared-error-mse/48788577#48788577) – desertnaut Jan 07 '19 at 14:14
  • Hmmmm... ok that makes sense... thanks for the link. So for a classification problem, depending on the application, someone might say that a 97% accuracy is good enough, or another may require a 99% accuracy... How do you know if you MSE is low enough for what you require, other than actually running the model and seeing how it performs? – DaveS Jan 07 '19 at 14:28
  • 2
    It depends on the task. But in general, just run the model, see the values of MSE and test the model. Then you will know how much more you have to achieve in MSE for your model to be good – Novak Jan 07 '19 at 14:34
  • @DaveS the above comment by Novak is correct; regarding the first part of your own comment - **no**: accuracy *does not even enter the discussion* for regression problems... – desertnaut Jan 07 '19 at 14:40

0 Answers0