Is the score from TransformedTargetRegressor of scikit-learn correct?

Question

I made a short Jupyter notebook to go with my question regarding the TransformedTargetRegressor.
I wanted to put a transformer inside a pipeline to play with a parameter grid but the scores didn't match.

...
model = linear_model.LinearRegression()
lg_tr = preprocessing.FunctionTransformer(func=np.log, inverse_func=np.exp, check_inverse=True)
y_log = lg_tr.transform(y)
score_0 = model.fit(X, y_log).score(X, y_log)

...
model = compose.TransformedTargetRegressor(func=np.log, inverse_func=np.exp, check_inverse=True,
    regressor=linear_model.LinearRegression())
score_1 = model.fit(X, y).score(X, y)

The score_0 value is correct. Why is the one from score_1 not?
I don't have problem with the prediction that works fine, only the score.
Did I miss something?
Thank you =)

I think score 1 is the right way to compute the performance in actual target value range — Venkatachalam, Jul 03 '21 at 01:25

score 1 · Accepted Answer · answered Jul 03 '21 at 08:39

Typically, you should be interested on how good your model performs (or scores) when predicting the actual values in their original range or scale. This is however what you are measuring with score_1 and not with score_0.

score_0 represents the model's performance when the target variable is in log scale which is in most cases not very useful.

score_1 however uses the score method of TransformedTargetRegressor which makes sure the target variable is in its original scale before computing any performance metric. Thus, judgements should be made based on score_1.

Thank you, that makes perfect sense. I was too focused on fitting the data and forgot the meaning... Have a good day =) — Gmoaltos, Jul 03 '21 at 09:05

Is the score from TransformedTargetRegressor of scikit-learn correct?

1 Answers1

Linked