3

I just made a Adaboost Classifier with these parameters,

1.n_estimators = 50

2.base_estimator = svc (support vector classifier)

3.learning_rate = 1

here is my code:

from sklearn.ensemble import AdaBoostClassifier
from sklearn.svm import SVC

svc = SVC(kernel = 'linear',probability = True)

ABC = AdaBoostClassifier(n_estimators = 50, base_estimator = svc, learning_rate = 1)

ABC.fit(X,Y)

Dataset has 18 independent variables and 1 categorical dependent variable dataset has 10480 datapoints

whenever i run this it will take so much time but no any result.

Is there any way to check execution time? Or any better way to do this?

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Dilumika Nawodya
  • 78
  • 5
  • why don't you try it with 1 or 2 estimators. if its fast, use more and see how it performs. Alternatively try it with less data to get a feeling of how fast oder slow your algorithm is. – Florian H Sep 17 '19 at 08:49
  • Yes i have tryed and it decreased the execution time as you sayed so thank you so much for your comment – Dilumika Nawodya Sep 22 '19 at 19:54

2 Answers2

6

In practice, we never use SVMs as base classifiers for Adaboost.

Adaboost (and similar ensemble methods) were conceived using decision trees as base classifiers (more specifically, decision stumps, i.e. DTs with a depth of only 1); there is good reason why still today, if you don't specify explicitly the base_classifier argument, it assumes a value of DecisionTreeClassifier(max_depth=1). DTs are suitable for such ensembling because they are essentially unstable classifiers, which is not the case with SVMs, hence the latter are not expected to offer much when used as base classifiers.

On top of this, SVMs are computationally much more expensive than decision trees (let alone decision stumps), which is the reason for the long processing times you have observed.

Unless you have a very good reason to stick to SVMs as base classifiers (and I highly doubt that you do), remove the base_estimator = svc in order to revert to the default setting, and most probably you will be fine.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
1

I had a similar experience recently. In my case though, I realised I wasn't scaling the X before using SVM as the base estimator. Just make sure you scale the data from 0 to 1 (you can use StandardScaler() from sklearn) which is always required prior to using SVM.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
crazy_daffodils
  • 21
  • 3