lightgbm how to deal with No further splits with positive gain, best gain: -inf

Question

how to deal with [Warning] No further splits with positive gain, best gain: -inf is there any parameters not suit?

Answer 1

Some explanation from lightGBM's issues:

it means the learning of tree in current iteration should be stop, due to cannot split any more.

I think this is caused by "min_data_in_leaf":1000, you can set it to a smaller value.

This is not a bug, it is a feature.

The output message is to warn user that your parameters may be wrong, or your dataset is not easy to learn.

link: https://github.com/Microsoft/LightGBM/issues/640

So on the contrary, the data is hard to fit.

Answer 2

This means that no improvement can be gained by adding additional leaves to the tree subject to the restrictions of the hyperparameters. It is not necessarily a bad thing, as limiting the depth of the tree can prevent overfitting. However, if the tree is underfitting the data, try tweaking these hyperparameters:

• decrease min_data_in_leaf - minimum number of data points in a leaf

• decrease min_sum_hessian_in_leaf - Minimum sum of the Hessian (second derivative of the objective function evaluated for each observation) for observations in a leaf. For some regression objectives, this is just the minimum number of records that have to fall into each node. For classification objectives, it represents a sum over a distribution of probabilities. It works like min_child_weight in xgboost.

• increase max_bin or max_bin_by_feature when creating dataset

LightGBM training buckets continuous features into discrete bins to improve training speed and reduce memory requirements for training. This binning is done one time during Dataset construction. Increasing the number of bins per feature can increase the number of splits that can be made.

max_bin controls the maximum number of bins that features will bucketed into. It is also possible to set this maximum feature-by-feature, by passing max_bin_by_feature.

Answer 3

set 'verbosity': -1 in params, it works!

Answer 4

Increase max_depth or set it to -1.