SVM Classification error when using my own kernel function - Matlab

Question

I'm trying to implement a svmtrain function. It works at training and it works when classifying sample matrix of same number of samples than the training matrix. What I am doing wrong?

MAIN FUNCTION

function [ output_args ] = RBFTest( xtrain, ytrain)
output_args=svmtrain(xtrain,ytrain,'Kernel_Function',@mine,...
'boxconstraint',1);
end

KERNEL FUNCTION (it supposes to be and RBF Kernel, I'm trying to accelerate using GPU)

function [ output ] = mine( U,V )
sig=1;
n=size(U,1);
K=U*V'/sig^2;
d=diag(K);
K=K-ones(n,1)*d'/2;
K=K-d*ones(1,n)/2;
output=exp(K);
end

I train....

>> R2=RBFTest(X(1:3000,:),Y(1:3000,:))

R2 = 

          SupportVectors: [3000x57 double]
                   Alpha: [3000x1 double]
                    Bias: -0.0219
          KernelFunction: @mine
      KernelFunctionArgs: {}
              GroupNames: [3000x1 double]
    SupportVectorIndices: [3000x1 double]
               ScaleData: [1x1 struct]
           FigureHandles: []

I try to classify a smaller part of the samples...but it doesn't work

>> mean(svmclassify(R2,X(1:2000,:))==Y(1:2000))
Error using svmclassify (line 114)
An error was encountered during classification.
Matrix dimensions must agree.

I try to classify same data used to train ... work!

>> mean(svmclassify(R2,X(1:3000,:))==Y(1:3000))

ans =

    0.9990

I retrain with smaller sample....

>> R2=RBFTest(X(1:1000,:),Y(1:1000,:))

R2 = 

          SupportVectors: [1000x57 double]
                   Alpha: [1000x1 double]
                    Bias: -0.0549
          KernelFunction: @mine
      KernelFunctionArgs: {}
              GroupNames: [1000x1 double]
    SupportVectorIndices: [1000x1 double]
               ScaleData: [1x1 struct]
           FigureHandles: []

I classify different samples but using same matrix and vectors size ... and it works

>> mean(svmclassify(R2,X(1001:2000,:))==Y(1001:2000))

ans =

    0.4610

Why it doesn't work when using different sizes of training and classification matrices?

score 0 · Answer 1 · edited May 23 '17 at 11:57

I don't think your RBF kernel function is correct (the part with the diag doesn't look right)...

Try the following vectorized implementation instead (see sources here and here):

function K = rbf(U, V, sigma)
    if nargin < 3, sigma = 1; end
    %K = pdist2(U, V, 'euclidean').^2;
    K = bsxfun(@plus, sum(U.^2,2), sum(V.^2,2).') - 2*(U*V');
    K = exp(-K ./ (2*sigma^2));
end

and use it as:

c = 1; gamma = 1;
model = svmtrain(Xtrain, Ytrain, 'Kernel_Function',@(u,v) rbf(u,v,gamma), ...
    'BoxConstraint',c);
Ytest_hat = svmclassify(model, Xtest);

In fact if you look at MATLAB's own private implementation of RBF kernel (which -all rbf_kernel), you'll see it is using a similar trick (only with repmat instead of the more efficient bsxfun)

SVM Classification error when using my own kernel function - Matlab

1 Answers1