How to combine OneHotEncoder (for strings) with Float features

Question

I'd like to use LogisticRegression to combine X features that are strings and floats. This question is similar to this question: Logistic regression on One-hot encoding

There is a comment:

I would like to add that your answer is partially correct. Indeed, if only LabelEncode the strings, and not one_hot encode them. That will create false results since some string will worth "more" than others. – Mornor Jun 27, 2017 at 7:19 2 If anyone is wondering what Mornor means, this is because label encode will be numerical values. Ex: France = 0, Italy = 1, etc. That means that some cities are worth more than others. With one-hot encoding each city has the same value: Ex: France = [1, 0], Italy = [0,1]. Also don't forget to the dummy variable trap algosome.com/articles/dummy-variable-trap-regression.html. – Juan Acevedo Jan 20,

However these are just comments. I would like to see the code that combines them as it's not intuitively obvious how to combine them.

Here is the code:

def build_model(results: List[Result]) -> Tuple[LogisticRegression, OneHotEncoder]:
    home_names = np.array([r.fixture.home_team.name for r in results])
    away_names = np.array([r.fixture.away_team.name for r in results])
    home_goals = np.array([r.home_goals for r in results])
    away_goals = np.array([r.away_goals for r in results])
    home_spis = np.array([r.home_spi for r in results])
    away_spis = np.array([r.away_spi for r in results])
    home_imps = np.array([r.home_imp for r in results])
    away_imps = np.array([r.away_imp for r in results])

    team_names = np.array(list(home_names) + list(away_names)).reshape(-1, 1)
    team_encoding = OneHotEncoder(sparse=False).fit(team_names)

    encoded_home_names = team_encoding.transform(home_names.reshape(-1, 1))
    encoded_away_names = team_encoding.transform(away_names.reshape(-1, 1))

    team_spis = np.array(list(home_spis) + list(away_spis)).reshape(-1, 1)
    home_spis_reshaped = np.array(list(home_spis) ).reshape(-1, 1)
    away_spis_reshaped = np.array(list(away_spis) ).reshape(-1, 1)



    x: NDArray[float64] = np.concatenate(
        [encoded_home_names, encoded_away_names, home_spis_reshaped, away_spis_reshaped], 1)  # type: ignore
    y = np.sign(home_goals - away_goals)

    model = LogisticRegression(penalty="l2", fit_intercept=False, multi_class="ovr", C=1)
    model.fit(x, y)

    return model, team_encoding

        if n_features != self.n_features_in_:
>           raise ValueError(
                f"X has {n_features} features, but {self.__class__.__name__} "
                f"is expecting {self.n_features_in_} features as input."
            )
E           ValueError: X has 1416 features, but LogisticRegression is expecting 1418 features as input.

../../env/lib/python3.10/site-packages/sklearn/base.py:400: ValueError

So it looks like I have to add the home/away spi float scores in before calling fit on the OneHotEncoder, but I'm unclear the best way to do this. Thanks

Solution based upon Alexander's help:

def build_model(results: List[Result]) -> Tuple[LogisticRegression, OneHotEncoder]:
home_names = np.array([r.fixture.home_team.name for r in results])
away_names = np.array([r.fixture.away_team.name for r in results])
home_goals = np.array([r.home_goals for r in results])
away_goals = np.array([r.away_goals for r in results])
home_spis = np.array([r.home_spi for r in results])
away_spis = np.array([r.away_spi for r in results])
home_imps = np.array([r.home_imp for r in results])
away_imps = np.array([r.away_imp for r in results])
team_names = np.array(list(home_names) + list(away_names)).reshape(-1, 1)

team_features = [home_names, away_names, home_spis, away_spis, home_imps, away_imps]

df = pd.DataFrame(team_features).transpose()
df.columns = ['home_team', 'away_team', 'home_spi', 'away_spi', 'home_importance', 'away_importance']
cat_columns = ["home_team", "away_team"]
model = LogisticRegression(penalty="l2", fit_intercept=False, multi_class="ovr", C=1)
team_encoding = OneHotEncoder(sparse=False).fit(team_names)
pipe = make_pipeline(
    ColumnTransformer(
        transformers=[
            ("encode", team_encoding, cat_columns),
        ],
        remainder="passthrough"
    ),
    SimpleImputer(),
    model
)

y = np.sign(home_goals - away_goals)
pipe = pipe.fit(df, y)
return model, team_encoding

Based upon Alexander's suggestions I came up with a working solution: — hockey_dave, Dec 22 '22 at 13:23

score 1 · Accepted Answer · answered Dec 21 '22 at 17:58

TL;DR: Don't reinvent pipelines, and don't do what that answer suggests (See also: why shouldn't I use LabelEncoder for categorical features?).

Here's a sample of the Titanic dataset that shows a mix of ordinal integer values (pclass), sex encoded as [male, female], and continuous age and fare variables:

      pclass     sex      age      fare
0          1  female  29.0000  211.3375
1          1    male   0.9167  151.5500
2          1  female   2.0000  151.5500
3          1    male  30.0000  151.5500
4          1  female  25.0000  151.5500

We'll OneHotEncode the categorical columns, impute missing values, and fit a LogisticRegression model as the last step in a pipeline. Notice the use of passthrough in the ColumnTransformer, which indicates that we should not apply a categorical encoding to the float features. MRE:

from sklearn.datasets import fetch_openml
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

titanic, target = fetch_openml(data_id=40945, parser="auto", return_X_y=True)
titanic = titanic[["pclass", "sex", "age", "fare"]]

cat_columns = ["sex", "pclass"]
cat_categories = [["male", "female"], [1, 2, 3]]

pipe = make_pipeline(
    ColumnTransformer(
        transformers=[
            ("encode", OneHotEncoder(categories=cat_categories), cat_columns),
        ],
        remainder="passthrough"
    ),
    SimpleImputer(),
    LogisticRegression()
)

X_train, X_test, y_train, y_test = train_test_split(titanic, target)

pipe = pipe.fit(X_train, y_train)
print(pipe.score(X_test, y_test))
# 0.7926

How to combine OneHotEncoder (for strings) with Float features

1 Answers1