May i use the sklearn predict_proba of a model (LogisticRegression for example) as a feature of another model (RandomForestClassifier for example)?

Question

I am trying to improve my classification model, using statsmodel in LogisticRegression i note that some features that didn't pass in t test and don't have many influency when i use this model are very important when i change the model, for example i looked up to feature_importances of a RandomForestClassifier and the more important feature did not influence LogisticRegression.

With this in mind, i thought to use LogisticRegression without this feature and use the predict_proba to pick the probabilities, then i create another model using RandomForest but now using all features and including the logisticRegressor probabilities. Or i can pick all probabilities of many models and use them as features of another model.. Anything of This make sense? I dont know if i am inserting any bias doing this and why.

You can think of a prior classifier as a feature extractor or transformer and it is quite convenient! — meti, Sep 14 '21 at 05:40
Please provide enough code so others can better understand or reproduce the problem. — Community, Sep 20 '21 at 08:33

score 0 · Answer 1 · answered Sep 21 '21 at 12:46

0

I found that what I was doing was stacking, but instead of using another model's response as a feature, I was using the probability of being 1 (predict_proba).

answered Sep 21 '21 at 12:46

Dept

1
1

May i use the sklearn predict_proba of a model (LogisticRegression for example) as a feature of another model (RandomForestClassifier for example)?

1 Answers1