ELKI, output: ROCAUC, Precision@k, f1.maximum

Question

What is Precision @ k used for in the outlier case? (when varying k on the same dataset I always get: Precision @ 3016 and I don't get where ELKI has that number from, number of outliers are 1508)

and

precision.average and precision.r?

and f1.maximum?

I know ROCAUC is a measure of how well the algorithm labels the outliers as outliers and the normal as normal objects.

I want to see if the quality of the outlier detection is good. Can I do that with the other measures too?

Computing LDOFs
LDOF for objects: 49534 [100%]
de.lmu.ifi.dbs.elki.algorithm.outlier.lof.LDOF.runtime: 116887 ms
Evaluating using minority class: yes
de.lmu.ifi.dbs.elki.evaluation.outlier.OutlierRankingEvaluation.rocauc: 0.736341684836717
de.lmu.ifi.dbs.elki.evaluation.outlier.OutlierRankingEvaluation.precision.average: 0.10795456476088741
de.lmu.ifi.dbs.elki.evaluation.outlier.OutlierRankingEvaluation.precision.r: 0.16578249336870027
de.lmu.ifi.dbs.elki.evaluation.outlier.OutlierRankingEvaluation.f1.maximum: 0.18336314847942753
ROCAUC: 0.7363416848367167
Precision @ 3016 0.13726790450928383

Answer 1

R-precision is Precision@k with k=number of outliers in your ground truth. At this point, recall = precision. So in your case, it would be precision@1508. The "r" comes from being the "recall = precision" point.

This is not the k parameter of the algorithm. But both are commonly called k in literature.

Maximum F1 is the maximum value of the F1 measure (of precision and recall) obtainable by varying the cutoff k. So there exists a k, where the harmonic mean of precision and recall is 0.18336.

Average precision is the usual average precision; evaluated at each outlier, then averaged.

For more detail, see e.g. https://en.wikipedia.org/wiki/Information_retrieval

Precision@2k is an artifact: the automatic evaluation would produce the precision@k curve for up to 2k (usually the interesting region). The curve would the output the precision at the end of the curve; but this is usually not particularly useful (you could manually configure the maximum k to clip the curve, but that doesn't make this an interesting k for evaluation). I will remove this from the classes; and also the second ROCAUC output is also not needed (also coming from the visual curve evaluation).