C/C++ Measuring Program Efficiency

Question

I thought the best way to measure how efficient my program is was by checking the time it took it to run however when I do so I keep getting random times.

I'm using the following code as shown by Roger Pate in this this thread:

#include <ctime>

void func()
{
    using namespace std;
    clock_t begin = clock();

    //Function to measure here

    clock_t end = clock();
    double elapsed_secs = double(end - begin) / CLOCKS_PER_SEC;
}

I think this is happening because I have a few programs running in the background.

I don't want to restart my computer every time I'd wanna recheck the time for my function so my question is: Is there a better way to measure a program's efficiency?

IDE is Codeblocks, OS is Win7 64bit.

Answer 1

With modern, multi-gigahertz CPUs, and high speed RAM I/O, unless your code takes more than a few minutes, or so, to run, measuring its execution time is unlikely to produce any meaningful results. The signal-to-noise ratio is just too high. I/O interrupts, and natural preemptive multitasking will produce too much noise, and drown out any meaningful runtime metrics.

Especially, as in your case, on MS Windows, which constantly kicks off various background O/S processes. And restarting your computer will only make things worse. The Windows O/S typically launches a gazillion background processes after a reboot, over the course of the next several minutes, that do one thing or another. Generally you need to wait 10-15 minutes, before natural background activity calms down.

Answer 2

You would need to measure the user time. You can accomplish this using the time command in UNIX based systems (https://en.wikipedia.org/wiki/Time_(Unix)) . I am not sure whether Microsoft has something similar.

User time is the time that the OS measures for your process so it is a more accurate depiction of how long your program has been running. It is able to do this because it is the entity scheduling tasks.

Answer 3

If you are concerned with the noise of I/O interrupts or preemptive multitasking, use GetThreadTimes function.

However, I would be more concerned with the resolution of the clock() function. On Windows, I would use QueryPerformanceCounter function.

Answer 4

As other's have already pointed out, using clock() is an imprecise way of measuring time, a better approach is to query the TSC counter directly. The TSC counter is extremely accurate, fast to query, and able to measure individual instruction execution speed (subject to a few constraints, below).

As an example of how to query the TSC, see the code below;

#ifdef WIN32
#define uint64_t unsigned __int64
#endif 

static inline uint64_t rdtsc (void) {
  static struct {
    union {
      struct { uint32_t hi; uint32_t lo; };
      uint64_t ret;
    };
  } tsc;
  asm volatile (
    "RDTSCP\n\t"
    "mov %%edx, %0\n\t"
    "mov %%eax, %1\n\t"
    : "=r" (tsc.lo), "=r" (tsc.hi)
    :: "%rax", "%rbx", "%rcx", "%rdx"
  );
  return tsc.ret;
}

// example: uint64_t start = rdtsc();
//          // Code goes here 
//          uint64_t end   = rdtsc();
//  printf("Code executed in %zd cycles\n", end-start);

The only caveat to using the TSC counter is that the TSC is only accurate on the core you are using, so for interrupt driven workloads, or long running processes, you are better off with querying the system time counter. A long time ago, the TSC counter was also influenced by the frequency the processor was running at.

The TLDR; Use the TSC for measuring the runtime of short snippets of code.

Answer 5

This is my first ever action here, so please be kind.

1) As mentioned before, for "lower" precision of the timer (around us) the function QueryPerformanceCounter() works fine. Additionally you should ask for the frequency this timer is stepping with QueryPerformanceFrequency(). Then you can determine the difference of ticks from begin to end and divide it by the frequency.

2) A higher resolution (also mentioned) is to directly read the TSC (TimeStampCounter) register of your machine. The command _rdtscp() is the best way to do so under windows (afaik). Here the difficulty is to determine the frequency your machine is currently running on. One approach is to measure the ticks used for a sleep(1000) and then determine the frequency by simply having the ticks difference as the frequency in Hz.

Example Code for 1)

#include "Windows.h"

_LARGE_INTEGER Start,Stop,Frequency;

DWORD64 Time_taken;

int YourFunction ()
{
 Start = QueryPerformanceCounter();

 // Run Code to measure here

 Stop = QueryPerformanceCounter();

 Frequency = QueryPerformanceFrequency();

 Time_taken = (Stop.QuadPart-Start.QuadPart)/Frequency.QuadPart;
 }

For 2)

 #include <intrin.h> 
 #include <stdio.h>

 unsigned __int64 Start,Stop,Frequency,Time_taken;   // Variables for measurement
 unsigned int test;    // Pointer-buffer for Machine AUX register (non-relevant)

 int YourFunction(void) 
 {
  // Measure current frequency
 Start = __rdtscp(&test);

 Sleep(1000);

 Stop = __rdtscp(&test);

 Frequency = Stop-Start;

 // Measure your function
 Start = __rdtscp(&test);

 // Code to measure

 Stop = __rdtscp(&test);

 Time_taken = (Stop-Start)/Frequency;
 }

For references see 1) https://msdn.microsoft.com/de-de/library/windows/desktop/ms644904%28v=vs.85%29.aspx

and 2) https://msdn.microsoft.com/de-de/library/bb385235.aspx

I hope this answers your question.

Greetings Sascha

Answer 6

Run your code under profiler, code could be optimized but with debug info.

Had good experience with VerySleepy: http://www.codersnotes.com/sleepy/