Stack Overflow
Stack Overflow

Questions tagged [yardstick]

Use this tag for questions relating to Yardstick, an open-source framework package for writing benchmarks.

Yardstick is an open-source framework package by GridGain Systems for writing benchmarks, and specifically writing benchmarks for clustered or otherwise distributed systems.

More information can be found at Yardstick GitHub and the GridGain website.

25 questions
4
votes
0 answers

different output for PR AUC for different R packages

I find different numeric values for the computation of the Area Under the Precision Recall Curve (PRAUC) with the dataset I am working on when computed via 2 different R packages: yardstick and caret. I am afraid I was not able to reproduce this…
deltasun
  • 314
  • 3
  • 11
4
votes
4 answers

Tidyverse syntax for calculating precision and recall

I am trying to calculate AUC, Precision, Recall, Accuracy for every group in my data frame (i have a single data frame that has predicted data from three different models concatenated). What is the tidyverse syntax to do it? I want to use the…
Abi K
  • 631
  • 2
  • 7
  • 13
3
votes
1 answer

Balanced log loss function in yardstick

Can someone help me figure out how to create a balanced logarithmic loss function in yardstick for use in a tidymodels pipeline? I looked up the documentation on creating custom metrics and I was able to create straightforward custom regression and…
JaredS
  • 242
  • 2
  • 5
  • 16
3
votes
1 answer

Modify figure sizes of `pr_curve` and `auc_curve` from R package yardstick

I'm trying to generate ROC curve and precision-recall curve using the library "yardstick". However, I could not find a way to modify the figure shape. Here's a toy example. ## Precision-recall curve data.frame(true = as.factor(rep(c(0,1), 10)), …
inmybrain
  • 386
  • 3
  • 16
2
votes
0 answers

tidymodels: f_meas metric_tweak error on metric_set

Using the iris dataset, a knn-classifier was tuned with iterative search for the purpose of multiple classification. However, an error is generated, when the macro-weighted version of f_meas (as created by metric_tweak) is used in metric_set. I…
bioblackgeorge
  • 45
  • 6
2
votes
1 answer

Add (or override) fill aesthetic to a ggplot2 autoplot function

I would like to add a fill aesthetic to an autoplot function. Specifically, to the autoplot.conf_mat function from the yardstick package. library(tidyverse) library(tidymodels) data("hpc_cv") cm <- hpc_cv %>% filter(Resample == "Fold01") %>% …
JasonAizkalns
  • 20,243
  • 8
  • 57
  • 116
2
votes
1 answer

tidymodels metric_set:Error: All inputs to `metric_set()` must be functions. These inputs are not: (2)

I have used recipe()function in tidymodels packages for imputation missing values and fixing imbalanced data. here is my data; mer_df <- mer2 %>% filter(!is.na(laststagestatus2)) %>% select(Id, Age_Range__c, Gender__c, numberoflead,…
Ozgur Alptekın
  • 505
  • 6
  • 19
2
votes
1 answer

How to set the estimate argument correctly for roc_auc() function in yardstick package

I want to compute the AUC for a model. library(yardstick) data(two_class_example) This code works. roc_auc( two_class_example, truth = truth, Class1, options = list(smooth = TRUE) ) I like to specify arguments so that my code is easier…
Anthony
  • 105
  • 7
2
votes
2 answers

yardstick::rmse on grouped data returns error and incorrect results

I wanted to evaluate the performance of several regression model and used the yardstick package to calculate the RMSE. Here is some example data model obs pred 1 A 1 1 2 B 1 2 3 C 1 3 When I run the following…
markus
  • 25,843
  • 5
  • 39
  • 58
1
vote
0 answers

Yardstick Benchmark in Apache Ignite

For performance evaluation, I'm using the yardstick benchmark on apache ignite. To increase the memory bandwidth while running the yardstick benchmark(ex.IgnitePutGetBenchmark), I directly modified the yardstick benchmark java source and compiled…
Yunkeuk Kim
  • 23
  • 3
1
vote
1 answer

Using Yardstick to calculate RMSE for aggregate of predictions per group

Sometimes I don't want to assess my models on their performance on predicting single observations, but rather I want to assess how a model performs for predictions in aggregate for groups. The group resampling tools in rsample, like group_vfold_cv,…
Econ_Modeler
  • 13
  • 3
1
vote
1 answer

Tidymodels prediction methods giving different results

I'm a bit confused about getting metrics from resamples using tidymodels. I seem to be getting 3 different metrics from the same set of resamples, depending on if I use collect_predictions() %>% metrics() or simply collect_metrics() Here is a simple…
Jeff
  • 57
  • 6
1
vote
0 answers

Ability to construct a custom metric in yardstick using multiple variables for multi class classification - tidymodels

I am transferring over from caret to the tidymodels environment and I am having an issue with transferring a custom metric. All the available documentation I can find provides examples of building a custom metric using the 'truth' and 'estimate'…
nealec
  • 21
  • 3
1
vote
2 answers

Tuning with classification_cost and custom cost matrix in Tidymodels

I am using tidymodels for building a model where false negatives are more costly than false positives. Hence I'd like to use the yardstick::classification_cost metric for hyperparameter tuning, but with a custom classification cost matrix that…
O René
  • 305
  • 1
  • 12
1
vote
1 answer

Get AUC on training data from a fitted workflow in Tidymodels?

I'm struggling with how the obtain the AUC from a logistic regression model using tidymodels. Here's an example using the built-in mpg dataset. library(tidymodels) library(tidyverse) # Use mpg dataset df <- mpg # Create an indicator variable for…
max
  • 4,141
  • 5
  • 26
  • 55
1
2