Is this overfitting?

Question

I have a CNN that is performing very well (96% accuracy, 1.~ loss) on training data but poorly (50% accuracy, 3.5 loss) on testing data.

enter image description here

If it's basically guessing on the test data, then yes. Try adding dropout and/or regularization layers. — TomNash, Jun 26 '19 at 20:34
@SamAtWork Excuse me.I change my model.And this is my result. https://imgur.com/u3aHvyc https://imgur.com/KeTPiLB I want to ask that is it overfitting? — Sherlock Lo, Jun 27 '19 at 15:50

score 0 · Answer 1 · answered Jun 26 '19 at 20:49

0

96% accuracy suggests you have a really close fit to your training data. 50% accuracy on testing data shows that your model cannot account for the noise/variability of the data being studied. This looks like textbook overfitting.

You seem to be calling your validation data your test data. Maybe you can better partition your data?

answered Jun 26 '19 at 20:49

Zach Favakeh

227
1
7

Should I change my model?Or add more data? – Sherlock Lo Jun 26 '19 at 21:02
First I would change your model. Maybe experiment with cross-validation. I don't know how much data you're using so I don't know about the last question. – Zach Favakeh Jun 26 '19 at 21:07
I will try it.Thanks. – Sherlock Lo Jun 26 '19 at 21:11
By the way my data is imbalance.Could you give me any suggestion? – Sherlock Lo Jun 26 '19 at 21:19
Here is a good guide I found for handling imbalanced data. : https://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/ – Zach Favakeh Jun 26 '19 at 21:22
excuse me.I change my model.And this is my result. https://imgur.com/u3aHvyc https://imgur.com/KeTPiLB I want to ask that is it overfitting? – Sherlock Lo Jun 27 '19 at 11:53
Good job, that looks much better than the original. It could be overfitting but it's hard to say. I would make a new post with your new model. – Zach Favakeh Jun 27 '19 at 14:59

score 0 · Accepted Answer · answered Jun 28 '19 at 10:15

The telltale signature of overfitting is when your validation loss starts increasing, while your training loss continues decreasing, i.e.:

(Image adapted from Wikipedia entry on overfitting)

Here are some other plots indicating overfitting (source):

See also the SO thread How to know if underfitting or overfitting is occuring?.

Clearly, your loss plot does exhibit such behavior, so yes, you are indeed overfitting.

On the contrary, the plot you have linked to in a comment:

does not exhibit such behavior, hence here you are not actually overfitting (you just have reached a saturation point, beyond which your validation error is not further improving).

Is this overfitting?

2 Answers2

Linked