0

To optimize the PR AUC (area under the precision-recall curve) in Caret, the prSummary function can be plugged into the trainControl function.

Is there a similar way to optimize the PRG AUC (area under the precision-recall-gain curve), as introduced by Flach and Kull? Inside or outside of Caret?

The MLeval package returns PRG curves and PRG AUCs, but only seems to run on the Caret train output.

Alexander
  • 72
  • 7
  • I trust these two answers can prove handy if you want to write your own summary: https://stackoverflow.com/questions/47240996/user-defined-summaryfunction-in-caret-logloss/47241523#47241523, https://stackoverflow.com/questions/52691761/additional-metrics-in-caret-ppv-sensitivity-specificity/52697940#52697940 – missuse Mar 21 '20 at 08:34
  • If you still cant get around making the custom summary when you have read the linked posts, comment here and i will answer. – missuse Mar 21 '20 at 09:21

1 Answers1

0

It turns out that Meelis Kull wrote an R package for the PRG AUC. Computing the PRG AUC is as simple as:

prg_curve <- create_prg_curve(labels, scores)
prg_auc <- calc_auprg(prg_curve)

For Caret, those functions can be plugged into a summary function, as explained by @missuse in the links in the comment above.

Alexander
  • 72
  • 7