Including Scaling and PCA as parameter of GridSearchCV

Question

I want to run a logistic regression using GridSearchCV, but I want to contrast the performance when Scaling and PCA is used, so I don't want to use it in all cases.

I basically would like to include PCA and Scaling as "parameters" of the GridSearchCV

I am aware I can make a pipeline like this:

mnl = LogisticRegression(fit_intercept=True, multi_class="multinomial")

pipe = Pipeline([
    ('scale', StandardScaler()),
    ('mnl', mnl)])

params_mnl = {'mnl__solver': ['newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga'],
              'mnl__max_iter':[500,1000,2000,3000]}

The thing is that, in this case, the scaling would be applied in all folds, right? Is there a way to make it so it's "included" in the gridsearch?

EDIT:

I just read this answer and even though it's similar to what I want, it's not really it, because in that case the Scaler will be applied to the best estimator out of the GridSearch.

What I want to do is, for example, let's say

params_mnl = {'mnl__solver': ['newton-cg', 'lbfgs']}

I want to run the regression with Scaler+newton-cg, No Scaler+newton-cg, Scaler+lbfgs, No Scaler+lbfgs.

See also https://stackoverflow.com/q/19262621/10495893 – Ben Reiniger Jun 07 '21 at 14:03 — Ben Reiniger, Jun 07 '21 at 14:03

jhihan · Accepted Answer · 2020-10-01T23:14:28.433

You can set up the parameters with_mean and with_std of StandardScaler() as False to represent no standerdization. In the GirdSearchCV, the parameter para_grid can be set up as

param_grid = [{'scale__with_mean': [False],
               'scale__with_std': [False],
               'mnl__solver': ['newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga'],
               'mnl__max_iter':[500,1000,2000,3000]
              },
              {'mnl__solver': ['newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga'],
              'mnl__max_iter':[500,1000,2000,3000]}
]

Then the first dict in the list is "No Scaler+mnl" and the second is "Scaler+mnl"

Ref:

https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html

https://scikit-learn.org/stable/tutorial/statistical_inference/putting_together.html

Edit: I think it's complicated if you also considering turn on/off PCA... Maybe you need to define a customised PCA which derives the original PCA. And then define additional boolean argument which determines whether the PCA should be executed or not...

class MYPCA(PCA):
    def __init__(self, PCA_turn_on, *args):
        super().__init__(*args)
        self.PCA_turn_on = PCA_turn_on
    
    def fit(X, y=None):
        if (PCA_turn_on == True):
            return super().fit(X, y=None)
        else:
            pass

    # same for other methods defined in PCA

that looks good! Thanks! Do you have any idea for the PCA tho? I couldn't find any parameter in it that could be set to False or anything — amestrian, Oct 01 '20 at 18:14
actually, I realized it's a bit more complicated than that. To use PCA I need to scale the data, so I would need for the case with PCA to be with Scaling as well, necessarily — amestrian, Oct 01 '20 at 19:37
I also think it's complicated... Maybe you need to define a customised PCA which derives the original PCA. And then define additional boolean argument which determines whether the PCA should be executed or not... — jhihan, Oct 01 '20 at 23:12
Tried doing what I could... even tried [this](https://stackoverflow.com/questions/42266737/parallel-pipeline-to-get-best-model-using-gridsearch) answer and it didn't quite do it... unfortunately I dont have much time to figure it out, so I'll do two grid searches, one with pca and one without, and then get the best model between the two — amestrian, Oct 01 '20 at 23:18

Ben Reiniger · Answer 2 · 2021-02-15T15:03:10.477

From the documentation for Pipeline:

A step’s estimator may be replaced entirely by setting the parameter with its name to another estimator, or a transformer removed by setting it to ‘passthrough’ or None.

For example:

pipe = Pipeline([
    ('scale', StandardScaler()),
    ('mnl', mnl),
])

params = {
    'scale': ['passthrough', StandardScaler()],
    'mnl__solver': ['newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga'],
    'mnl__max_iter': [500, 1000, 2000, 3000],
}

Including Scaling and PCA as parameter of GridSearchCV

2 Answers2