1

I have a number-crunching program where the equations I want to solve are represented by the member functions of different classes. Each class of course has several member variables which are inputs to the equations. The member vars are currently primitives like double and int, but for better integration with a GUI, I want to replace the primitives with managed variables; i.e. I want to use a separate class to hold the variable's name and value, and to handle reading and writing its value. I am concerned about both performance and readability of the code. E.g., I would rather see "natural" looking code like x = y + 2 rather than x.set_value(y.get_value() + 2).

I came up with four different ways of doing this and experimented with how much time each method takes (code below). I compiled this with MSVC 2013 using a debug build. I get nonsense results in release mode because I think my loops get optimized away. The results seem significant; using primitives or accessing member variables directly takes half the time of using getter/setter functions or cast operator overloads.

My questions are: Am I testing these different methods appropriately? And is there a better way to do what I'm trying to do? Thanks.

#include <iostream>
#include <chrono>

using namespace std;
using namespace std::chrono;

//Class to manage input parameters
struct Parameter {
    double _value = 0.0;
    double Get_value() const {return _value;}
    void Set_value(double value) {_value = value;}
    operator double(){return _value;}
    void operator=(const double& rhs) {_value = rhs;}
};

int main() {
    const size_t NUM_TESTS = 100;       //Number of tests to run
    const size_t MAX_ITER = 1000000;    //Number of iterations to run in each test
    const double x = 2.71828;           //Variable to read from
    double y = 0;                       //Variable to write to
    Parameter test_parameter;       //managed variable to read/write from/to
    double test_primitive = 0.0;    //primitive variable to read/write from/to
    size_t t_primitive = 0;         //Total time spent on primitive variable (microseconds)
    size_t t_managed_cast = 0;      //Time spent on managed variable using cast and assignment operators
    size_t t_managed_getset = 0;    //Time spent on managed variable using getter/setter functions;
    size_t t_managed_direct = 0;    //Time spent on managed variable using direct access of member var.

    for (size_t n = 0; n < NUM_TESTS; ++n) {
        //Test using a primitive variable.
        auto t0 = high_resolution_clock::now();
        for (size_t i = 0; i < MAX_ITER; ++i) {
            test_primitive = x;
            y = test_primitive;
        }
        auto t1 = high_resolution_clock::now();
        t_primitive += duration_cast<microseconds>(t1-t0).count();

        //Test using a managed variable, using cast operator and assignment operator
        t0 = high_resolution_clock::now();
        for (size_t i = 0; i < MAX_ITER; ++i) {
            test_parameter = x;
            y = test_parameter;
        }
        t1 = high_resolution_clock::now();
        t_managed_cast += duration_cast<microseconds>(t1-t0).count();

        //Test using managed variable, using getter/setter member functions
        t0 = high_resolution_clock::now();
        for (size_t i = 0; i < MAX_ITER; ++i) {
            test_parameter.Set_value(x);
            y = test_parameter.Get_value();
        }
        t1 = high_resolution_clock::now();
        t_managed_getset += duration_cast<microseconds>(t1-t0).count();

        //Test using managed variable, using direct public access
        t0 = high_resolution_clock::now();
        for (size_t i = 0; i < MAX_ITER; ++i) {
            test_parameter._value = x;
            y = test_parameter._value;
        }
        t1 = high_resolution_clock::now();
        t_managed_direct += duration_cast<microseconds>(t1-t0).count();
    }

    cout << "Average time for primitive (microseconds): " << t_primitive / NUM_TESTS << endl;
    cout << "Average time for managed with cast (microseconds): " << t_managed_cast / NUM_TESTS << endl;
    cout << "Average time for managed with get/set (microseconds): " << t_managed_getset / NUM_TESTS << endl;
    cout << "Average time for managed with direct access (microseconds): " << t_managed_direct / NUM_TESTS << endl;

    return 0;
}
Carlton
  • 4,217
  • 2
  • 24
  • 40
  • *I think my loops get optimized away* ... see: [Loop with a zero execution time](http://stackoverflow.com/q/26771692/1708801) – Shafik Yaghmour Feb 10 '15 at 19:12
  • Your intuition is correct, there is overhead in calling functions. If you are really concerned about performance, directly accessing and modifying the member variable is faster than using functions to do so. However note that getters and setters, specifically, are heavily optimized by almost all compilers – Cory Kramer Feb 10 '15 at 19:13
  • 7
    @Cyber: There is overhead to a runtime function call, but a function call in source code doesn't mean a runtime call. That is after all, the point of inlining. **The right way to deal with meaningless benchmark results is not to disable optimization, but to fix the benchmark.** – Ben Voigt Feb 10 '15 at 19:14
  • The *may* be overhead if those calls exist to finality within the compiled code you're actually going to *run* (which I have to assume will not be *debug* code). Non-virtual const-getters are easily one of the most frequently inlined optimizations made by compilers. Fix you release-code so it is meaningful and bench correctly. – WhozCraig Feb 10 '15 at 19:16
  • 1
    @BenVoigt I agree, and didn't mean to suggest otherwise (if I did!). If you are concerned about the performance of optimized code, you should profile against optimized code, there is no question about that. – Cory Kramer Feb 10 '15 at 19:16
  • You won't run non-optimized builds on the real data, so benchmarks not using optimization are meaningless. Heed Ben Voigt's advice and make it so you can benchmark with optimizations enabled. (I wish I had a comprehensive resource to link to at this point, but benchmarking seems to be a dark art.) –  Feb 10 '15 at 19:16
  • I figured using debug was wrong. Now I know that `volatile` isn't just for embedded systems! – Carlton Feb 10 '15 at 19:29

2 Answers2

3
volatile const double x = 2.71828;
volatile double y = 0;
// ^^ VERY IMPORTANT

There, I fixed your benchmark.

Now enable optimization again, and worry only about timing with optimization enabled.

Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • I had a suspicion that I needed a `volatile` in there somewhere. Curiously, the compiler didn't accept my assignment operator for the volatile variable. I had to overload it like this: `void operator=(volatile const double& rhs) {_value = rhs;}` Edit: enabling optimizations changed everything. There is virtually no difference between any of the four methods now. Thanks Ben. – Carlton Feb 10 '15 at 19:26
  • 1
    @Carlton: Better yet would be `double operator=(double rhs) { return _value = rhs; }` Primitive types should almost always be passed by value, not const reference. – Ben Voigt Feb 10 '15 at 19:29
1

A debug build will not inline those accessor methods, so it will take longer. A release build will not have this problem.

Try using a release build but make your variables volatile, I believe that will disable the loop optimization.

rlbond
  • 65,341
  • 56
  • 178
  • 228