1

Is it possible to train a multiclass (multinomial) linear classification model with Vowpal Wabbit Library?

I tried to use --oaa with --loss_function squared, but it seems that the default loss function for --oaa is logistic.

I am using rcv1.multiclass as input.

One Solution:

I can create multiple version of the data as follows:

Version i: make all the labels zero except class I

Then I can train multiple binary classifications for each version of data. Finally, I can feed the test data to all the classifier and apply an argmax. Is there any better (automated) solution?

AmirC
  • 326
  • 5
  • 14
  • Does this answer your question? [PurgeCSS and Tailwind CSS, how to preserve responsive classes using the Command Line Interface?](https://stackoverflow.com/questions/65554596/purgecss-and-tailwind-css-how-to-preserve-responsive-classes-using-the-command) – cantdutchthis Mar 21 '21 at 05:55

1 Answers1

2

When you use vw --oaa N, you will actually get a linear N-class classifier. To get a non-linear classifier you would need to add quadratic/polynomial features (-q, --cubic, --interactions) or kernels (--ksvm) or a hidden layer (--nn) or any other nonlinear reduction (--lrq, --stage_poly, --autolink).

The choice of loss function does not effect whether the classifier is linear or not. The default is --loss_function=squared. For classification, I would suggest to use --loss_function=logistic (possibly with --probabilities if you want to predict probability of each class) or --loss_function=hinge (if you care only about the top class).

Then I can train multiple binary classifications for each version of data. Finally, I can feed the test data to all the classifier and apply an argmax. Is there any better (automated) solution?

Yes, this is exactly what --oaa does (but more efficiently).

Martin Popel
  • 2,671
  • 12
  • 22