3

I need to implement multiple classification classifier using Liblinear. Accord.net machine learning framework provides all of Liblinear properties except the Crammer and Singer’s formulation for multi-class classification. This is the process.

Suhaib Janjua
  • 3,538
  • 16
  • 59
  • 73

1 Answers1

3

The usual way of learning a multi-class machine is by using the MulticlassSupportVectorLearning class. This class can teach one-vs-one machines that can then be queried using either voting or elimination strategies.

As such, here is an example on how linear training can be done for multiple classes:

// Let's say we have the following data to be classified
// into three possible classes. Those are the samples:
// 
double[][] inputs =
{
    //               input         output
    new double[] { 0, 1, 1, 0 }, //  0 
    new double[] { 0, 1, 0, 0 }, //  0
    new double[] { 0, 0, 1, 0 }, //  0
    new double[] { 0, 1, 1, 0 }, //  0
    new double[] { 0, 1, 0, 0 }, //  0
    new double[] { 1, 0, 0, 0 }, //  1
    new double[] { 1, 0, 0, 0 }, //  1
    new double[] { 1, 0, 0, 1 }, //  1
    new double[] { 0, 0, 0, 1 }, //  1
    new double[] { 0, 0, 0, 1 }, //  1
    new double[] { 1, 1, 1, 1 }, //  2
    new double[] { 1, 0, 1, 1 }, //  2
    new double[] { 1, 1, 0, 1 }, //  2
    new double[] { 0, 1, 1, 1 }, //  2
    new double[] { 1, 1, 1, 1 }, //  2
};

int[] outputs = // those are the class labels
{
    0, 0, 0, 0, 0,
    1, 1, 1, 1, 1,
    2, 2, 2, 2, 2,
};

// Create a one-vs-one multi-class SVM learning algorithm 
var teacher = new MulticlassSupportVectorLearning<Linear>()
{
    // using LIBLINEAR's L2-loss SVC dual for each SVM
    Learner = (p) => new LinearDualCoordinateDescent()
    {
        Loss = Loss.L2
    }
};

// Learn a machine
var machine = teacher.Learn(inputs, outputs);

// Obtain class predictions for each sample
int[] predicted = machine.Decide(inputs);

// Compute classification accuracy
double acc = new GeneralConfusionMatrix(expected: outputs, predicted: predicted).Accuracy;

You can also try to solve a multiclass decision problem using the one-vs-rest strategy. In this case, you can use the MultilabelSupportVectorLearning teaching algorithm instead of the multi-class one shown above.

Cesar
  • 2,059
  • 25
  • 30
  • 1
    Thank you so much. I tested it and it perfectly worked. sorry but I have just another question that is so important to me. how can I implement it in a parallel way? – vahid ziaiyan Apr 19 '15 at 08:34
  • 1
    Hmmm... The learning phase is already done in parallel. The multi-class problem is decomposed as a series of binary decision problems, and each binary decision is solved in a separate thread. If this is not happening for you, then please let me know, and I will investigate! – Cesar Apr 19 '15 at 12:41