Running functions in R multiple times during benchmarking

Question

I am trying to measure the computation time of a function in R using system.time(). I want to run the function a few hundred times to get an average but I don't want to copy and paste that many times. Is there an easier way to do that?

Ari B. Friedman · Accepted Answer · 2011-09-06T16:09:35.450

17

The microbenchmark package takes a ,times= option and has the added bonus of being a bit more accurate.

> library(microbenchmark)
> m <- microbenchmark( seq(10)^2, (1:10)^2, times=10000)
> m
Unit: nanoseconds
       expr   min    lq median    uq     max
1  (1:10)^2  2567  3423   3423  4278   41918
2 seq(10)^2 44484 46195  46195 47051 1804147
> plot(m)

plot microbenchmark

And using the not-yet-released autoplot() method for ggplot2:

autoplot(m)

autoplot microbenchmark

edited Sep 06 '11 at 16:09

answered Aug 02 '11 at 22:31

Ari B. Friedman

71,271
35
175
235

I like to plot it in lattice with either `require(lattice); bwplot(expr~time, m, auto.key=TRUE, xlim=quantile(v$time,c(0,0.95)))` or `require(latticeExtra); ecdfplot(~time, groups=expr, m, auto.key=TRUE, xlim=quantile(v$time,c(0,0.95)))`. – Marek Aug 03 '11 at 09:27
@Marek I agree the default plot looks terrible.... When I used it the other day in an actual example I used `ggplot` and did quartiles: http://stackoverflow.com/questions/1296646/how-to-sort-a-dataframe-by-columns-in-r/6871968#6871968 – Ari B. Friedman Aug 03 '11 at 09:34
Just a note that `autoplot.microbenchmark` as used above is now available in `taRifx` 1.0.3, which should be available on CRAN within the next few hours. – Ari B. Friedman May 31 '12 at 23:40

score 12 · Answer 2 · answered Aug 02 '11 at 21:43

12

system.time(replicate (  ... stuff ..) )

Or: (hey, I'm not ashamed to have the same answer as Dirk.)

require(rbenchmark)
benchmark( stuff... )   # Nice for comparative work

answered Aug 02 '11 at 21:43

IRTFM

258,963
21
364
487

score 11 · Answer 3 · answered Aug 02 '11 at 21:42

You want to use the rbenchmark package and its function benchmark() which does just about everything for you.

Here is the first example from its help page:

R> example(benchmark)

bnchmrR> # example 1
bnchmrR> # benchmark the allocation of one 10^6-element numeric vector, 
bnchmrR> # replicated 100 times
bnchmrR> benchmark(1:10^6)
    test replications elapsed relative user.self sys.self user.child sys.child
1 1:10^6          100   0.327        1      0.33        0          0         0

For truly expression-level benchmarking, there is also the microbenchmark package.

Running functions in R multiple times during benchmarking

3 Answers3

Linked