2

I'm trying to implement a one class SVM using the e1071 package in R. Can somebody give me pointers on how to optimize the F-score using a grid search ?

I have tried the tune.svm functions but it has only resulted in high sensitivity or high Specificity.

The percentage of positive class which I'm trying to predict is about 1-2% in the general population.

The results i get have high accuracy but with a very low F-score:

             Reference
Prediction    members Not members
  members           1           4
  Not members      12         983

           Accuracy : 0.984           
             95% CI : (0.9741, 0.9908)
No Information Rate : 0.987           
P-Value [Acc > NIR] : 0.83691         

              Kappa : 0.1046

Mcnemar's Test P-Value : 0.08012 

        Sensitivity : 0.07692         
        Specificity : 0.99595

Here are snippets of my code:

tuned <- tune.svm(fo, data = df, 
                  nu =  0.001:0.5,
                  gamma = 10^(-2:0), 
                  type='one-classification'                 
                  );

model <-     svm(fo, data = df , 
                   nu = tuned$best.parameters$nu, 
                   gamma = tuned$best.parameters$gamma,
                   type='one-classification'
                 );
ML_Passion
  • 1,031
  • 3
  • 15
  • 33

1 Answers1

1

You can provide the tunecontrol parameter of the tune.svm method.

It takes an object of the class tunecontrol.

You then need to create the object with the desired parameters. By specifying the error.fun parameter when building this object you can define the error function to be used.

error.fun : function returning the error measure to be minimized. It takes two arguments: a vector of true values and a vector of predicted values. If NULL, the misclassification error is used for categorical predictions and the mean squared error for numeric predictions.

So putting a function which computes the F-Score should do the job.

Community
  • 1
  • 1
alexandrekow
  • 1,927
  • 2
  • 22
  • 40
  • I changed my code to add a function for calculating F-score and also modified the tune function: tuned <- tune.svm(fo, data = df, nu = 2^(-5:-0.5), gamma = 2^(-6:0), #cost = 2^(-3:0), tunecontrol = tune.control(sampling = "cross",performances = TRUE, error.fun = FscoreErr(y,prediction)), type='one-classification' ) , where FscoreErr is the error function , don't know how to the prediction vector when grid search executes in tune. – ML_Passion Mar 30 '15 at 16:33
  • Sorry since i am relatively new to stack overflow, don't know how to put my entire code in the comments here. – ML_Passion Mar 30 '15 at 16:34
  • `tuned <- tune.svm(fo, data = df,` `nu = 2^(-5:-0.5), gamma = 2^(-6:0),` `tunecontrol = tune.control(sampling = "cross",performances = TRUE, error.fun = FscoreErr(y,prediction)),` `type='one-classification' ` ` ) – ML_Passion Mar 31 '15 at 03:12