Weka : how to use cross validation in code

Question

I was trying to use cross validation in following code: Program:

   TextDirectoryToArff d = new TextDirectoryToArff();

      try {
    Instances dataset = d.createDataset("C:\\mytest");
    dataset.setClassIndex(dataset.numAttributes() - 1 );

    double precision = 0, recall=0,fmeasure=0,error=0;

    int size1 = dataset1.numInstances() / 10;

    int begin = 0;
    int end = size1 - 1 ;

    for (int i=1 ; i<=10;i++)
    {
        System.out.println("iteration :" + 1);

        Instances training = new Instances(dataset);
        Instances testing = new Instances(dataset, begin , (end - begin));

        for (int j=0;j < (end - begin); j++)
            training.delete(begin);

        Classifier tree = new NaiveBayes();

        Instances filteredInstaces = training;
        StringToNominal nominal ;

        for(int a=0;a<training.numAttributes()-1;a++)
        {
            if(training.attribute(a).isString())
            {
                nominal = new StringToNominal();

                nominal.setInputFormat(filteredInstaces);
                training = Filter.useFilter(training, nominal);
            }
        }

        tree.buildClassifier(training);

        Evaluation eval = new Evaluation(testing);

        eval.evaluateModel(tree, testing);
        System.out.println("Precision:" + eval.precision(1));
        System.out.println("Recall:" + eval.recall(1));
        System.out.println("Fmeasure:" + eval.fMeasure(1));
        System.out.println("Error:" + eval.errorRate());

I've some code for cross validation but not able to integrate with above code. Please suggest how can I integrate following code in above code to find cross validation?

Code:

 Evaluation eval = new Evaluation(dataset);
 eval.evaluateModel(cls, dataset2);
 eval.crossValidateModel(cls,dataset1,10, dataset2.getRandomNumberGenerator(1));   
 System.out.println(eval.toSummaryString("\nResults\n======\n", false));

you can find more info about cross validation concept at http://stackoverflow.com/questions/10437677/cross-validation-in-weka — Istvan Nagy, Mar 16 '16 at 17:01

Istvan Nagy · Answer 1 · 2016-03-16T17:03:14.203

I think you are confused a bit about weka Evaluation.crossValidateModel method. It already calculate the 10 different train and test fold, and train 10 models on trains and evaluate the models on test, so not necessary to calculate it as you tried in your code.

so in your code: TextDirectoryToArff d = new TextDirectoryToArff();

  try {
Instances dataset = d.createDataset("C:\\mytest");
dataset.setClassIndex(dataset.numAttributes() - 1 );

// you already have a dataset

Classifier naiveBayes = new NaiveBayes();
//you need a classifier
Evaluation eval = new Evaluation(dataset);
//only call the crossValidateModel with your classifier, on your dataset, with 10 fold, and random
eval.crossValidateModel(naiveBayes, dataset, 10, new Random(1));
//print the results of 10 fold
System.out.println(classifier);
System.out.println(eval.toSummaryString());
System.out.println(eval.toMatrixString());
System.out.println(eval.toClassDetailsString());

you can find more at https://weka.wikispaces.com/Generating+cross-validation+folds+(Java+approach)

Weka : how to use cross validation in code

1 Answers1