7

According to http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html I read:

"The fit time complexity is more than quadratic with the number of samples which makes it hard to scale to dataset with more than a couple of 10000 samples."

I have currently 350,000 samples and 4,500 classes and this number will grow further to 1-2 million samples and 10k + classes.

My problem is that I am running out of memory. All is working as it should when I use just 200,000 samples with less than 1000 classes.

Is there a way to build-in or use something like minibatches with SVM? I saw there exists MiniBatchKMeans but I dont think its for SVM?

Any input welcome!

domi771
  • 450
  • 1
  • 9
  • 21
  • Possible duplicate of [Does the SVM in sklearn support incremental (online) learning?](http://stackoverflow.com/questions/23056460/does-the-svm-in-sklearn-support-incremental-online-learning) – Mikhail Korobov Nov 22 '16 at 11:14
  • Just out of curiosity, why does it have 10k+ classes? Sounds like too many classes. – aerin Oct 04 '17 at 23:32

1 Answers1

4

I mentioned this problem in my answer to this question.

You can split your large dataset into batches that can be safely consumed by an SVM algorithm, then find support vectors for each batch separately, and then build a resulting SVM model on a dataset consisting of all the support vectors found in all the batches.

Also if there is no need in using kernels in your case, then you can use sklearn's SGDClassifier, which implements stochastic gradient descent. It fits linear SVM by default.

Community
  • 1
  • 1
Sergey Zakharov
  • 1,493
  • 3
  • 21
  • 40