Testing warnings in R by comparing strings (best idea?)

Question

I'm working on an R package and I need some help writing R test functions that are meant to check whether the correct warning is being thrown on C-side code and then caught on the R side. Let me give you some background on what I'm working on exactly:

1. Most of what I'm writing is done on the C side. In addition, I have an if-statement type macro in C that allows the coder to pass a warning to R in the form of a string. The basic premise is that if(statement_true) pass_warning_to_R("Warning string to pass"). What I'd like to do is test whether these warnings are being thrown when I expect/need them to be by writing an R file that uses tryCatch blocks.

2. So far I've written something similar to this:

   counter <- 0
   tryCatch({
   
   function_im_testing()
   
   }, warning = function(war) {
        # Check if warning is as expected and if so increment counter
        if(toString(war)=="The warning I'm expecting/testing for"){
           print(toString(war))
           counter <- counter + 1
        }
   
   }, error = function(err) {
         print(toString(err))
   
   }, finally = {
   print("Leaving tryCatch")
   })
   
   # Stop if the 3 warnings we expected aren't present
   stopifnot(counter == 3)
   

This is the method I'm using and, so far, I haven't even been able to get the if statement to execute by trying to get toString(war) and "Warning I'm expecting/testing for" to be the same thing. This, in addition with the fact that this method is pretty sloppy and unreliable, leads me to believe that there's a better way. So, is there a better approach to doing this?

Answer 1

Usually with warnings you'd like to allow evaluation to continue; tryCatch is used to stop evaluation. So instead use withCallingHandlers with a handler for warnings that does what you want, and then invokes the 'muffleWarning' restart. The message of an error / warning can be extracted with conditionMessage

counter <- 0L
withCallingHandlers({
    function_im_testing()
}, warning = function(w) {
    if (conditionMessage(w) == "The warning I'm expecting/testing for")
        counter <<- counter + 1L
    invokeRestart("muffleWarning")
})

Since you're writing your own package, it makes sense to create warnings that can be identified in a more robust way, e.g., the following returns a condition that can be used in warning, but that has a class 'bad_input' that can be used in withCallingHandlers.

bad_input <- function(...) {
    w <- simpleWarning(...)
    class(w) <- c("bad_input", class(w))
    w
}

To be used like warning(bad_input("your input is bad")) and producing output with

fun <- function() {
    warning("oops")
    warning(bad_input("your input is bad"))
    "DONE"
}        

like

>     fun()
[1] "DONE"
Warning messages:
1: In fun() : oops
2: your input is bad 
>     counter <- 0L
>     withCallingHandlers(fun(), bad_input = function(w) {
+         counter <<- counter + 1L
+         invokeRestart("muffleWarning")
+     })
[1] "DONE"
Warning message:
In fun() : oops
> counter
[1] 1

Answer 2

Apart from actually capturing the warning, you need to be aware that warning messages are translated:

library(devtools)
with_envvar(c(LANG = "en"), log(-1))
# In log(-1) : NaNs produced
with_envvar(c(LANG = "de"), log(-1))
# In log(-1) : NaNs wurden erzeugt
with_envvar(c(LANG = "fr"), log(-1))
# In log(-1) : production de NaN
with_envvar(c(LANG = "ko"), log(-1))
# In log(-1) : NaN이 생성되었습니다

So if you're doing this inside a test, make sure you set the LANG environmental variable to ensure that the message doesn't vary according to what computer it's run on.

Answer 3

Check out testthat::expect_warnings()