How to special-case the number of iterations in google benchmark?

Question

I am aware of the --benchmark_repetitions flag and it is not what I need. I want to be able to specify the number of iterations for one benchmark. I am okay with using a --benchmark_iterations flag which sets the number of iterations for all benchmarks.

I know google benchmark is smart to figure out how many iterations is needed to get a good measurement. This is good enough for most use cases but my use case is different. I need to be able to control the number of iterations precisely.

Harper · Answer 1 · 2021-02-13T21:58:42.507

9

BENCHMARK(YourBenchmark)->Iterations(num_ite); will do the trick.

You can specify iterations for each benchmark.

If you are using BENCHMARK_F, do that in the constructor

class BenchmarkBase : public benchmark::Fixture {
public:

    BenchmarkBase() {
        Iterations(num_ite);
    }
};

edited Feb 13 '21 at 21:58

answered Feb 13 '21 at 19:18

Harper

1,794
14
31

Or benchmark::RegisterBenchmark(...)->Iterations(num_ite) for custom main()'s. – mcandre Apr 26 '21 at 23:15
I tried adding `Iterations(num_ite)` in either constructor or `SetUp()` but nether worked. – Yixing Liu Feb 16 '22 at 05:04
FWIW -- adding it to `SetUp()` did not work, but adding it to the constructor *did* work for me. – Russ Tedrake Mar 25 '22 at 11:47

score 2 · Answer 2 · answered May 19 '20 at 10:32

It doesn't support the case of wanting to tune the iteration count precisely.

As you've pointed out, it runs until there are enough iterations to get a good enough signal (though it doesn't yet actually check the statistics). We used to have a way to set the iteration counts, but the interaction with the timing related flags are complex and were confusing to people trying to configure their benchmarks, so they were removed.

It is something that comes up often though, and if there is a way we could support iterations and timing together, or support iterations but also warn if we think the result isn't meaningful, then i'm open to considering it.

score 1 · Answer 3 · answered May 12 '23 at 19:42

Since version 1.8.0, Google Benchmark now support specifying the number of iteration directly from the command line. This is done through the --benchmark_min_time option. Simply specify the number of iteration followed by x.

For example, to run all benchmark with only one iteration each:

--benchmark_min_time=1x

The change was made here: https://github.com/google/benchmark/pull/1525

How to special-case the number of iterations in google benchmark?

3 Answers3