1

I've trained an XGBoost model using Sklearn, and I'm extracting feature importance using the eli5 package. I noticed that all my feature importances were positive:

For example:

enter image description here

How would I distinguish between features that have a positive/negative effect on my response variable if this kind of output is yielded?

Sandro
  • 29
  • 1
  • 3

2 Answers2

0

Since not all features are shown in the image you provided, I'll assume that you don't have a lot more features.

There is not any particular reason why you need to assume that there are features that have a negative impact on your result. If all the features have a positive effect (as shown in this output) you can just keep using them.

If your goal is to do feature selection based on the importance that you've extracted, you can choose a threshold to keep features based on their weight. This threshold will most likely need some trial and error to optimise your results.

But if this is not your goal, I see no apparent reason to distinguish any features here.

0

For xgboost, show_weights shows feature importance, which is always positive (and eli5+xgboost support three possible importance measures). If you want something like the average direction of the relationship between the feature and target, you'll need a different tool.

Ben Reiniger
  • 10,517
  • 3
  • 16
  • 29