1

I used Keras in Python to create a regression model for deep learning. After validation by cross-validation with k=5, train_loss averaged around 0.10 over 5 trials and validation_loss averaged around 0.12.

I thought it would be natural that validation_loss would be larger due to outliers in the data (or values that are just at the upper or lower limits).

However, I don't know how much validation_loss should deviate from train_loss to be considered as overfitting.

If you have any ideas on the criteria for determining whether or not there is overlearning, what else I should check, or if I have a fundamental difference in thinking, could you tell me please.

Here is example figure about loss value. (showing 2 figures as examples from 5 figures) enter image description here enter image description here

TR123
  • 117
  • 1
  • 6
  • In your graphs, see how the validation loss is starting to rise even though the training loss is still decreasing? That indicates overfitting. The learning rate might be the easiest parameter to help tune this, but it's usually a data / #parameters problem – en_Knight Jul 26 '21 at 03:59
  • To be clear, I think it's pretty subtle, the validation loss is, if anything, rising pretty slowly. But I think it's clear it isn't *decreasing*, would you agree? – en_Knight Jul 26 '21 at 03:59
  • Thank you for your advice. You think when epoch=200, train loss is decreasing, on the other hand, test loss is increasing, so this model is overfits don't you? I am learning machine learning on my own, so I didn't know the real criteria. I understood your think. Thank you very much. – TR123 Jul 26 '21 at 04:09
  • No problem! Yes, overfiting is always tricky to diagnose, but in general, if training loss is decreasing while testing loss does not, then you're only "memorizing the training set", not learning the deeper surface. Randomizing your training/testing split will help, if you haven't done that. Getting more data is the classic (though sometimes impractical) advice. Otherwise, you can try picking a smaller model, using some kind of "regularization" tool, or tweaking hyperparameters like learning rate – en_Knight Jul 26 '21 at 04:16

1 Answers1

0

I obtained the idea to solve this question, thanks to en_Knight.

TR123
  • 117
  • 1
  • 6