-1

So I have a simple Sequential model built in Keras and I'm asked to train it multiple times (particularly, 5 times) with the same set of data (although I would have to change the train-test split). I would like to, then, perform an average of these trained models in the sense of:

  1. Average the final accuracy on train and on validation.
  2. Average the learning curve.

I know I can do this with a loop using plain python and stuff, but since this seems to me a common thing to do, I wonder if there is already a built in function to do exactly that. I think there is a way to train multiple times and save the best model, but I just want to do the average of the final results.

Aliara
  • 41
  • 4
  • 2
    Unfortunately you are wrong, this is not a common thing to do, and there is no built-in function to do that. – Dr. Snoopy Jul 29 '20 at 10:33

2 Answers2

0

Maybe you are thinking about Bagging. https://en.wikipedia.org/wiki/Bootstrap_aggregating

You can train multiple models and average their outputs, but not the models. There isn't and there won't be a function for that because it is as simple as for example calculating average in regression task.

Mateusz Puto
  • 78
  • 7
0

For averaging accuracy metrics over n trials, see the pseudocode below

def loop_model(x_train, y_train, x_test, n_loops = 5):
        arr_metric = list()
        for _ in range(n_loops):
          print("Trial ", _+1 ," of ", n_loops)
          model = build_model(units, activation, ...)
          history = model.fit(x_train, y_train, ...)
          y_true, y_pred = model.predict(x_test)
          metric = compute_metric(y_true, y_pred)
          arr_metric.append(metric)
          del model
          keras.backend.clear_session()
        # Convert to numpy array to compute mean
        avg_metric = np.array(arr_metric).mean() 
        return avg_metric

The function build_model() compiles the model while compute_metric() computes your accuracy metric. These are not built-in functions but a part of pseudocode. Using a numpy array to compute the mean is one approach and there are other things you can try.

See this answer as to why I suggested using the last two lines in the for loop.

Kartik
  • 33
  • 1
  • 7