RandomForestRegressor in sklearn giving negative scores

Question

I'm surprised that i get a negative score on my predictions using the RandomForestRegressor, I'm using the default scorer(coefficient of determination). any help will be appreciated. my dataset looks something like this. dataset screenshot here

from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.model_selection import cross_val_score,RandomizedSearchCV,train_test_split
import numpy as np,pandas as pd,pickle
dataframe = pd.read_csv("../../notebook/car-sales.csv")
y = dataframe["Price"].str.replace("[\$\.\,]" , "").astype(int)
x = dataframe.drop("Price" , axis = 1)
cat_features = [
    "Make",
    "Colour",
    "Doors",
]
oneencoder = OneHotEncoder()
transformer = ColumnTransformer([
("onehot" ,oneencoder, cat_features)
],remainder="passthrough")
transformered_x = transformer.fit_transform(x)
transformered_x = pd.get_dummies(dataframe[cat_features])
x_train , x_test , y_train,y_test = train_test_split(transformered_x , y , test_size = .2)
regressor = RandomForestRegressor(n_estimators=100)
regressor.fit(x_train , y_train)
regressor.score(x_test , y_test)

I'm curious why did you overwrite `transfomered_x` value on `transformered_x = transformer.fit_transform(x)` with `transformered_x = pd.get_dummies(dataframe[cat_features])`? Also I don't think you need to use both, perhaps this (https://stackoverflow.com/questions/36631163/what-are-the-pros-and-cons-between-get-dummies-pandas-and-onehotencoder-sciki) might be useful — strivn, Jun 18 '20 at 08:46
Please **clarify** - are you getting a negative score (as you say in the title) or negative predictions (as you say in the body)? If the latter, please explain why negative predictions are an issue here (regression can give both positive and negative outputs). — desertnaut, Jun 18 '20 at 12:55
@IvanWiryadi i was using the get dummies to test to see if incase the transformer was the source of the problem. but just assume i never wrote the get_dummies line — Duncan, Jun 26 '20 at 20:31

score 3 · Accepted Answer · answered Jun 22 '20 at 03:40

I modified your code just a little bit and was able to achieve a score of 89%. You were SO close! Nicely done on your part. Not shabby!

from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.model_selection import train_test_split
import pandas as pd
dataframe = pd.read_csv("car-sales.csv")
df.head()
y = dataframe["Price"].str.replace("[\$\.\,]" , "").astype(int)
x = dataframe.drop("Price", axis=1)
cat_features = ["Make", "Colour", "Odometer", "Doors", ]
oneencoder = OneHotEncoder()
transformer = ColumnTransformer([("onehot", oneencoder, cat_features)], remainder="passthrough")
transformered_x = transformer.fit_transform(x)
transformered_x = pd.get_dummies(dataframe[cat_features])

x_train, x_test, y_train, y_test = train_test_split(transformered_x, y, test_size=.2, random_state=3)

forest = RandomForestRegressor(n_estimators=200, criterion="mse", min_samples_leaf=3, min_samples_split=3, max_depth=10)

forest.fit(x_train, y_train)

# Explained variance score: 1 is perfect prediction
print('Score: %.2f' % forest.score(x_test, y_test, sample_weight=None))
print(forest.score(x_test, y_test))

I think it was negative due to extreme over-fitting due to an extremely small amount of data.

This is directly from the sklearn documentation:

and I quote the document:

https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html

https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html

The coefficient R^2 is defined as (1 - u/v), where u is the residual sum of 
squares ((y_true - y_pred) ** 2).sum() and v is the total sum of squares 
((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it 
can be negative (because the model can be arbitrarily worse). A constant model 
that always predicts the expected value of y, disregarding the input features, 
would get a R^2 score of 0.0.

I enlarged the dataset to 100 rows, dropped the surrogate key (first column having int id 0-99) and here it is:

RandomForestRegressor in sklearn giving negative scores

1 Answers1