scikit-learn cross validation, negative values with mean squared error

Question

When I use the following code with Data matrix X of size (952,144) and output vector y of size (952), mean_squared_error metric returns negative values, which is unexpected. Do you have any idea?

from sklearn.svm import SVR
from sklearn import cross_validation as CV

reg = SVR(C=1., epsilon=0.1, kernel='rbf')
scores = CV.cross_val_score(reg, X, y, cv=10, scoring='mean_squared_error')

all values in scores are then negative.

Yes, this is supposed to happen. I forget exactly why, but I believe it's related to them minimizing the result when performing grid searching. The actual MSE is simply the postive version of the number you're getting. — David, Jan 29 '14 at 23:44
possible duplicate of [sklearn GridSearchCV with Pipeline](http://stackoverflow.com/questions/21050110/sklearn-gridsearchcv-with-pipeline) -- @David is right, when the unified scoring API was introduced, we decided to always maximize the score, which means scores that are actually losses need to be negated. — Fred Foo, Jan 30 '14 at 09:51

score 89 · Accepted Answer · edited May 23 '17 at 12:10

89

Trying to close this out, so am providing the answer that David and larsmans have eloquently described in the comments section:

Yes, this is supposed to happen. The actual MSE is simply the positive version of the number you're getting.

The unified scoring API always maximizes the score, so scores which need to be minimized are negated in order for the unified scoring API to work correctly. The score that is returned is therefore negated when it is a score that should be minimized and left positive if it is a score that should be maximized.

This is also described in sklearn GridSearchCV with Pipeline.

edited May 23 '17 at 12:10

Community

1
1

answered Dec 05 '14 at 19:36

AN6U5

2,755
22
20

Thanks for this. But while going for the best model, the negative MSE's are not considered while selecting the best model. It just takes the smallest in the positive MSE's for the best models. Any way to get around this? Thanks in advance! – Arpan Mar 12 '19 at 19:00
2

Thanks! One question, if I've gotten [-44, -33, -22] as mse scores just think of them as mse =44, 33, 22 ? – haneulkim Mar 09 '20 at 02:16

score 3 · Answer 2 · answered May 13 '20 at 17:33

You can fix it by changing scoring method to "neg_mean_squared_error" as you can see below:

from sklearn.svm import SVR
from sklearn import cross_validation as CV

reg = SVR(C=1., epsilon=0.1, kernel='rbf')
scores = CV.cross_val_score(reg, X, y, cv=10, scoring='neg_mean_squared_error')

score 1 · Answer 3 · answered Jan 24 '21 at 08:23

1

To see what are available scoring keys use:

import sklearn
print(sklearn.metrics.SCORERS.keys())

You can either use 'r2' or 'neg_mean_squared_error'. There are lots of options based on your requirement.

answered Jan 24 '21 at 08:23

MD Rijwan

471
6
15

scikit-learn cross validation, negative values with mean squared error

3 Answers3

Linked

Related