QuickCheck-Like Template Function Benchmarking in C++

Question

Inspired by Haskell's elegant way of auto-generating (random) instances of a given type, for example in QuickCheck, I'm trying to figure out how to write an as-easy-to-use-as-possible benchmarking framework in C++. I figure I will be using function templates, possibly with the help of new features in C++11 such as variadic templates. My hope is that I only need to specify a function or even better a function template and an STL template container type (an in turn its value_type) compatible with the argument(s) of the function.

I figured, that benchmarking a function with a set of inputs of various size is somewhat similar to how threads are setup and spawned in C++11. My first try would be to copy the constructor of the thread class and turn it into a benchmark function as

template< class Function, class ...Args >
inline
void benchmark( Function&& f, Args&&... args );

I'm not sure whether we should use r-value refs here or not. However f and args must be explicitly instantiated prior to the call to benchmark resulting in cumbersome non-functional use.

This lead to me to try to skip call arguments and only use template arguments instead:

namespace pnw
{
    template <template <typename> class Function, typename Container>
    inline
    void benchmark_container()
    {
        Function<typename Container::iterator> f;
        Container c(10);
        f(c.begin(), c.end());
    }
}

called as

typedef int T;
typedef std::vector<T> C;
pnw::benchmark_container<std::sort, C>();

However, compilation now errors as

tests/t_histogram.cpp: In function ‘void test_benchmark()’:
tests/t_histogram.cpp:56:44: error: no matching function for call to ‘benchmark_container()’
tests/t_histogram.cpp:56:44: note: candidate is:
tests/../benchmark.hpp:32:6: note: template<template<class> class Function, class Container> void pnw::benchmark_container()

I'm not sure whether C++ can handle transferring function template solely over the template arguments of another calling function.

Is this the right way to do it or is this not possible in C++11? I'm using GCC-4.6.

Answer 1

If you need to support "higher-kinded" parameters, you have to use template-template parameters. Also, inside a template, f::g will be considered a value if not qualified by typename. Therefore you should write:

template <template <typename> class Function, typename Container>  // <--
inline void benchmark_container()
{
    Function<typename Container::iterator> f;   // <--
    ...

(All of these are available before C++11.)

---

Edit: But the call

benchmark_container<std::sort, C>();

won't work, because std::sort is an overloaded template function, not a class template. You cannot refer to std::sort alone either, because it would be ambiguous.

If you only want to work with functions like std::sort which has no associated contexts, you could pass a function pointer to disambiguate with overload to use:

template <typename Container,
          void (*func)(typename Container::iterator, typename Container::iterator)>
inline void benchmark_container()
{
    Container c (10);
    func(c.begin(), c.end());
}

benchmark_container<std::vector<int>, std::sort>();

or

template <typename Container>
inline void benchmark_container(void (*func)(typename Container::iterator, typename Container::iterator))
{
    Container c (10);
    func(c.begin(), c.end());
}

benchmark_container<std::vector<int>>(std::sort);

or just manually choose which overload you'd like to use, allowing general function objects to be passed:

template <typename Container, typename F>
inline void benchmark_container(const F& function)
{
    Container c (10);
    function(c.begin(), c.end());
}

benchmark_container<std::vector<int>>(std::sort<std::vector<int>::iterator>);