2

I am trying to learn RankSVM using OHSUMED dataset and SVM Rank library as explained in following link: http://research.microsoft.com/en-s/um/beijing/projects/letor/Baselines/RankSVM-Struct.txt

I used same parameters as link suggests for OHSUMED dataset. i.e

OHSUMED/QueryLevelNorm/cv_l1_e0.001/fold1_l1_c0.0002_e0.001.log OHSUMED/QueryLevelNorm/cv_l1_e0.001/fold2_l1_c0.002_e0.001.log OHSUMED/QueryLevelNorm/cv_l1_e0.001/fold3_l1_c0.01_e0.001.log OHSUMED/QueryLevelNorm/cv_l1_e0.001/fold4_l1_c0.02_e0.001.log OHSUMED/QueryLevelNorm/cv_l1_e0.001/fold5_l1_c0.01_e0.001.log

But when I train my model & run "svm_rank_classify" command I get following result:

Reading model...done.
Reading test examples...done.
Classifying test examples...done
Runtime (without IO) in cpu-seconds: 0.00
Average loss on test set: 0.3864
Zero/one-error on test set: 100.00% (0 correct, 22 incorrect, 22 total)
NOTE: The loss reported above is the fraction of swapped pairs averaged over
all rankings. The zero/one-error is fraction of perfectly correct rankings!
Total Num Swappedpairs : 31337
Avg Swappedpairs Percent: 38.64

Please suggest If any steps I am missing here?

Thanks.

Rohan Surdikar
  • 31
  • 5
  • Can you clarify your question some? Are you just asking what the zero/one-error means and why it's not less than 100%? Were you expecting different output? – dmh Nov 04 '15 at 14:34

1 Answers1

1

The zero/one-error is the percentage of rankings (i.e. qid sets) where the model ranked at least one pair incorrectly. Your accuracy over all pairs is actually:

(100 - Avg Swappedpairs Percent) = 61.36%
dmh
  • 1,053
  • 12
  • 26