Variance thresholding shows RuntimeWarning: Degrees of freedom <= 0 for slice

Question

I have a dataframe of features and I'd like to remove low variance features. I used the following function to filter out low variance columns:

def variance_threshold_selector(data, threshold):

    selector = VarianceThreshold(threshold)
    selector.fit(data)

return data[data.columns[selector.get_support(indices=True)]]

And I got the following warning for different threshold values:

/anaconda3/envs/Observation/lib/python3.7/site-packages/sklearn/feature_selection/_variance_threshold.py:77: RuntimeWarning: Degrees of freedom <= 0 for slice.
  self.variances_ = np.nanvar(X, axis=0)

/anaconda3/envs/Observation/lib/python3.7/site-packages/sklearn/feature_selection/_variance_threshold.py:88: RuntimeWarning: invalid value encountered in less_equal
  (self.variances_ <= self.threshold)):

/anaconda3/envs/Observation/lib/python3.7/site-packages/sklearn/feature_selection/_variance_threshold.py:99: RuntimeWarning: invalid value encountered in greater

What are these warnings and how can I resolve them?

can you create some reproducible example using `sklearn.datasets`? — Venkatachalam, Mar 03 '20 at 16:10
Does this answer your question? [Mean of empty slice and Degrees of freedom <=0](https://stackoverflow.com/questions/43301112/mean-of-empty-slice-and-degrees-of-freedom-0) — PV8, Mar 04 '20 at 06:46

score 0 · Accepted Answer · answered Mar 04 '20 at 06:45

following this post:

RuntimeWarning: Degrees of freedom <= 0 for slice

occurs when you use the wrong shape

following this post:

RuntimeWarning: invalid value encountered in less_equal

hat's most likely happening because of a np.nan somewhere in the inputs involved

and following this post:

RuntimeWarning: invalid value encountered in greater

Your problem is caused by the NaN or Inf elements in your out_vec array

Variance thresholding shows RuntimeWarning: Degrees of freedom <= 0 for slice

1 Answers1