0

I have a function which is written using R and later a C++ version is also created. I want to evaluate if these two functions are producing same results or not.

The function generates random numbers and therefore needs to be compared under same seed.

Is this possible to compare a function written in 2 different language?

Rel_Ai
  • 581
  • 2
  • 11
  • giving an input and checking if the output is the same, is not a good way to check if two function are the same – Alberto Sinigaglia May 03 '21 at 17:22
  • @Berto99: It's not a bad way, especially if the original is a black box. See [test oracle](https://en.wikipedia.org/wiki/Test_oracle). Combined with fuzz testing, it can be pretty effective (as always if you know your domain you can fashion more unit tests). – AndyG May 03 '21 at 17:28
  • could you do not set the seed prior to calling each function and cache the results of each? then do `identical(res1, res2)` – Justin Landis May 03 '21 at 17:30
  • Does this answer your question? [Creating a PRNG engine for in C++11 that matches PRNG results in R](https://stackoverflow.com/questions/53201392/creating-a-prng-engine-for-random-in-c11-that-matches-prng-results-in-r) – Peter O. May 03 '21 at 17:33
  • @AndyG yes, it increases probability, but does not certify anything... but in case we can't prove that the two functions are the same, it's the best way the one you describe – Alberto Sinigaglia May 03 '21 at 17:33
  • See also: https://stackoverflow.com/questions/64965398/when-and-why-are-random-number-generators-not-sampled-for-certain-functions-and ; https://stackoverflow.com/questions/43221681/changing-rs-seed-from-rcpp-to-guarantee-reproducibility . Also, in general, the best way to "sync" PRNGs between two programs in different languages is to implement the same PRNG in both languages. – Peter O. May 03 '21 at 17:34
  • The problem is that it is impossible to prove that two functions ALWAYS provide the same result through testing alone. It is necessary to ALSO justify a claim that the set of test cases is sufficient to provide that verification - and that justification cannot be done solely through testing. Testing is only as good as the argument that the test cases are sufficient. – Peter May 03 '21 at 18:43

2 Answers2

2

The original question is underspecified: what exactly do you need to do?

We do give you the same set of random draws, witness this:

> library(Rcpp)
> set.seed(42); runif(2); rnorm(2)
[1] 0.914806 0.937075
[1] -0.564698  0.363128
> Rcpp::cppFunction("NumericVector myrunif(int n) { return Rcpp::runif(2); }")
> Rcpp::cppFunction("NumericVector myrnorm(int n) { return Rcpp::rnorm(2); }")
> set.seed(42); myrunif(2); myrnorm(2)
[1] 0.914806 0.937075
[1] -0.564698  0.363128
>

Not how both the U(0,1) and N(0,1) draws are identical because Rcpp accesses the same underlying RNG stream as R.

Dirk Eddelbuettel
  • 360,940
  • 56
  • 644
  • 725
1

The function you want to test in R , can be invoked from C++ in a fairly simple way (Rcpp)

Some pseudocode below.

// [[Rcpp::export]]
NumericVector GetRSideRandomValue(){

    // calling R function() -- randNumOnR
    Function f("randNumOnR");   

    // Return result.
    return f();
}

you can directly call GetRSideRandomValue on c++ side.

Then compare the values.

Pavan Chandaka
  • 11,671
  • 5
  • 26
  • 34
  • In fact this seems like something what I did. I am calling the C++ function with Rcpp and running my function in R under the same seed. However, I am getting a different set of numbers. Can you elaborate the part where you are calling R function inside C++ function? – Rel_Ai May 03 '21 at 18:04
  • Why would you do that? Why not just call the R function from R, and export the C++ via Rcpp to R, and call both? We have used that exact same pattern in probably a thousand Rcpp answers just here on StackOverflow. – Dirk Eddelbuettel May 03 '21 at 19:01
  • @DirkEddelbuettel Yep the other way is also fine. But depends on in which programming language the test bed is. – Pavan Chandaka May 03 '21 at 19:45
  • @DirkEddelbuettel Basically after `set.seed()`, I called both the R function and C++ function. But It's not working (giving different result based on different set of random numbers). I am not sure if I understood if this is what you are asking to do. – Rel_Ai May 04 '21 at 05:45
  • @Rel_Ai We would need a [minimally complete verifiable example](https://stackoverflow.com/help/minimal-reproducible-example) to help you. – Dirk Eddelbuettel May 04 '21 at 12:37