Stack Overflow
Stack Overflow

Questions tagged [ensemble-learning]

In statistics and machine learning, ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms.

In statistics and machine learning, ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms. Unlike a statistical ensemble in statistical mechanics, which is usually infinite, a machine learning ensemble refers only to a concrete finite set of alternative models, but typically allows for much more flexible structure to exist between those alternatives.

Use this tag and the appropriate language/library tag to ask questions about implementations of ensemble learning.

445 questions
25
votes
2 answers

What is a weak learner?

I want to compare different error rates of different classifiers with the error rate from a weak learner (better than random guessing). So, my question is, what are a few choices for a simple, easy to process weak learner? Or, do I understand the…
Stu
  • 1,543
  • 3
  • 17
  • 31
18
votes
4 answers

Ensemble of different kinds of regressors using scikit-learn (or any other python framework)

I am trying to solve the regression task. I found out that 3 models are working nicely for different subsets of data: LassoLARS, SVR and Gradient Tree Boosting. I noticed that when I make predictions using all these 3 models and then make a table of…
Maksim Khaitovich
  • 4,742
  • 7
  • 39
  • 70
16
votes
5 answers

Using sklearn voting ensemble with partial fit

Can someone please tell how to use ensembles in sklearn using partial fit. I don't want to retrain my model. Alternatively, can we pass pre-trained models for ensembling ? I have seen that voting classifier for example does not support training…
shubhamgoel27
  • 1,391
  • 10
  • 17
16
votes
4 answers

sklearn.ensemble.AdaBoostClassifier cannot accecpt SVM as base_estimator?

I am doing a text classification task. Now I want to use ensemble.AdaBoostClassifier with LinearSVC as base_estimator. However, when I try to run the code clf = AdaBoostClassifier(svm.LinearSVC(),n_estimators=50, learning_rate=1.0, …
allenwang
  • 727
  • 2
  • 8
  • 25
11
votes
2 answers

How to handle categorical variables in sklearn GradientBoostingClassifier?

I am attempting to train models with GradientBoostingClassifier using categorical variables. The following is a primitive code sample, just for trying to input categorical variables into GradientBoostingClassifier. from sklearn import datasets from…
Zhiyong
  • 391
  • 1
  • 3
  • 19
10
votes
1 answer

Building ensemble model with the python merf library

I would like to use the merf (mixed effect random forest) library in an ensemble model e.g. by using the mlens or mlxtend python libraries. However, due to the non-traditional way in which the fit and predict methods of merf are structured, I am…
user308827
  • 21,227
  • 87
  • 254
  • 417
10
votes
1 answer

Run multiple models of an ensemble in parallel with PyTorch

My neural network has the following architecture: input -> 128x (separate fully connected layers) -> output averaging I am using a ModuleList to hold the list of fully connected layers. Here's how it looks at this point: class…
MasterScrat
  • 7,090
  • 14
  • 48
  • 80
10
votes
1 answer

Custom learner function for Adaboost

I am using Adaboost to fit a classification problem. We can do the following: ens = fitensemble(X, Y, 'AdaBoostM1', 100, 'Tree') Now 'Tree' is the learner and we can change this to 'Discriminant' or 'KNN'. Each learner uses a certain Template…
WJA
  • 6,676
  • 16
  • 85
  • 152
9
votes
3 answers

R Caret Package Error - At least one of the class levels is not a valid R variable name

I am receiving the following error in R when stacking using the caret package. "Error: At least one of the class levels is not a valid R variable name; This will cause errors when class probabilities are generated because the variables names will…
aistow
  • 91
  • 1
  • 1
  • 4
9
votes
1 answer

Why is my VotingClassifier accuracy less than my individual classifier?

I am trying to create an ensemble of three classifiers (Random Forest, Support Vector Machine and XGBoost) using the VotingClassifier() in scikit-learn. However, I find that the accuracy of the ensemble actually decreases instead of increasing. I…
sfactor
  • 12,592
  • 32
  • 102
  • 152
8
votes
2 answers

Homogeneous vs heterogeneous ensembles

I would like to check with you if my understanding about ensemble learning (homogeneous vs heterogeneous) is correct. Is the following statement correct? An homogeneous ensemble is a set of classifiers of the same type built upon different data as…
Zoya
  • 1,195
  • 2
  • 12
  • 14
8
votes
3 answers

How to merge keras sequential models with same input?

I am trying to create my first ensemble models in keras. I have 3 input values and a single output value in my dataset. from keras.optimizers import SGD,Adam from keras.layers import Dense,Merge from keras.models import Sequential model1 =…
Eka
  • 14,170
  • 38
  • 128
  • 212
8
votes
1 answer

r caretEnsemble warning: indexes not defined in trControl

I have some r/caret code to fit several cross-validated models to some data, but I'm getting a warning message that I'm having trouble finding any information on. Is this something I should be concerned…
Max Power
  • 8,265
  • 13
  • 50
  • 91
8
votes
4 answers

GradientBoostingClassifier with a BaseEstimator in scikit-learn?

I tried to use GradientBoostingClassifier in scikit-learn and it works fine with its default parameters. However, when I tried to replace the BaseEstimator with a different classifier, it did not work and gave me the following error, return y -…
iampat
  • 1,072
  • 1
  • 12
  • 23
7
votes
2 answers

Is there a way to ensemble two keras (h5) models trained for same classes

I have trained two keras models with different datasets for same class labels. How could I ensemble the models keras_model.h5 and keras_model2.h5 together and make another keras model say keras_ensemble.h5. I have tried referring various internet…
Suriya
  • 113
  • 1
  • 8
1
2 3
…
29 30