3

I don't understand the meaning of bias parameter in the API of LIBLINEAR. Why is it specified by user during the training? Shouldn't it be just a distance from the separating hyperplane to origin which is a parameter of the learned model?

This is from the README:

struct problem
{
    int l, n;
    int *y;
    struct feature_node **x;
    double bias;
};

If bias >= 0, we assume that one additional feature is added to the end of each data instance.

What is this additional feature?

ks1322
  • 33,961
  • 14
  • 109
  • 164
lizarisk
  • 7,562
  • 10
  • 46
  • 70

1 Answers1

5

Let's look at the equation for the separating hyperplane:

w_1 * x_1  + w_2 * x_2  + w_3 * x_3 + ... + w_bias * x_bias = 0

Where x are the feature values and w are the trained "weights". The additional feature x_bias is a constant, whose value is equal to the bias. If bias = 0, you will get a separating hyperplane going through the origin (0,0,0,...). You can imagine many cases, where such a hyperplane is not the optimal separator.

The value of the bias affects the margin through scaling of w_bias. Therefore the bias is a tuning parameter, which is usually determined through cross-validation similar to other parameters.

M456
  • 5,547
  • 2
  • 19
  • 14
  • 1
    I still don't get it. I thought that the equation looks like "sum(w_i * x_i) + bias", where bias is learned from the training data. But we have w_bias and x_bias. You wrote: "The additional feature x_bias is a constant, whose value is equal to the bias". Do you mean that x_bias = bias? And w_bias is learned? Should I add additional feature x_bias to the feature vector of training data? – lizarisk Apr 27 '13 at 11:59
  • Yes, `bias = x_bias`. All w are learned parameters. No, you don't need to explicitly append anything to your feature vectors. – M456 Apr 27 '13 at 22:58