Why AES code in Crypto++ gives different performance results?

Question

I am trying to test performance of AES encryption. But whenever I am running the code it is giving different results.Why? Here's the code in C++ using Crypto++:

 int main(int argc, char* argv[]){
AutoSeededRandomPool prng;

byte key[AES::DEFAULT_KEYLENGTH];
prng.GenerateBlock(key, sizeof(key));

byte iv[AES::BLOCKSIZE];
prng.GenerateBlock(iv, sizeof(iv));

CBC_Mode< AES >::Encryption e;
e.SetKeyWithIV(key, sizeof(key), iv);

CBC_Mode< AES >::Decryption d;
d.SetKeyWithIV(key, sizeof(key), iv);

Time testing is here:

  clock_t startTime, finishTime;    
std::string plain = "AES CBC Test";   
std::string cipher, encoded, recovered;   
startTime = clock();    
try
{

    // The StreamTransformationFilter removes
    //  padding as required.
    StringSource s(plain, true, 
        new StreamTransformationFilter(e,
            new StringSink(cipher)
        ) // StreamTransformationFilter
    ); // StringSource

}
catch(const CryptoPP::Exception& e)
{
    cerr << e.what() << endl;
    exit(1);
}    
    // save current time just after finishing the encryption loop
finishTime = clock();

and my test results are here:

enter code heredouble executionTimeInSec = double( finishTime - startTime ) / CLOCK_TICKS_PER_SECOND;    

std::cout << "Encryption loop execution time: " << executionTimeInSec * 1000.0 << " microseconds." << std::endl;

std::cout << "Plain text size: " << plain.size() << " bytes." << std::endl;

double data_rate_MiBps = ((double)plain.size() / 1048576) / ((double)executionTimeInSec) ;

std::cout << "Encryption/decryption loop execution time MB/S: " << data_rate_MiBps << " MB/S." << std::endl; 
return 0;}

Timing unoptimized debug build. Compiled result1:

Encryption loop execution time: 0.041 microseconds.

Compiled result2:

Encryption loop execution time: 0.057 microseconds.

It would help your chances of an answer if your code actually compiled. — Jesper Juhl, Mar 02 '17 at 20:29
You should also post whether you are timing an optimized, release build, or an unoptimized "debug" build. If it is the latter, then these timings are meaningless. — PaulMcKenzie, Mar 02 '17 at 20:58
Also see [How can I run in Crypto++ library benchmarks test?](http://stackoverflow.com/q/29264531/608639) and [Speed difference between AES/CBC encryption and decryption?](http://stackoverflow.com/q/20164502/608639) — jww, Mar 02 '17 at 21:16
Your code is incomplete; in particular, it seems to be missing at least one `#include`. Please [edit] your code so it's a [mcve] of your problem, then we can try to reproduce and solve it. You should also read [ask]. — Toby Speight, Mar 03 '17 at 13:35
Also see [Benchmarks](https://www.cryptopp.com/wiki/Benchmarks) on the Crypto++ wiki and [Calculate time encryption of AES/CCM in Visual Studio 2017](https://stackoverflow.com/q/47956337/608639) on Stack Overflow. — jww, Dec 25 '17 at 07:40

score 1 · Accepted Answer · answered Mar 02 '17 at 23:54

0.041 microseconds is too short a timeframe to test in. To get a reliable measure you need to perform many iteration of your test and then divide the total time by the number of iterations you did.

When measuring in so short time frames many factors might mess up your timings:

The resolution of the clock on your system might not be high enough giving relative big jumps in your measures.
Your timing only measure elapse time, not the actual time spent running on a CPU. The impact of the OS assigning your CPU to something else in one test as opposed to another introduces big swings in the measure. When doing many iterations you smooth this random impact out on many iterations and thus removes the impact of chance.
Etc.

jww · Answer 2 · 2017-03-09T12:13:37.243

Related to Crypto++ benchmarking, the library supplies a benchmark suite in cryptest.exe. You invoke it like below. The b means benchmark; the 3 means run the test for 3 seconds; the 2.4 means the cpu frequency is 2.4 GHz.

./cryptest.exe b 3 2.4

The command produces output similar to below.

You can find the source code at bench1.cpp and bench2.cpp. For AES, you want to check bench2.cpp. Here's the code responsible for generating the numbers:

void BenchMark(const char *name, StreamTransformation &cipher, double timeTotal)
{
    const int BUF_SIZE=RoundUpToMultipleOf(2048U, cipher.OptimalBlockSize());
    AlignedSecByteBlock buf(BUF_SIZE);
    Test::GlobalRNG().GenerateBlock(buf, BUF_SIZE);

    unsigned long i=0, blocks=1;
    double timeTaken;

    clock_t start = ::clock();
    do
    {
        blocks *= 2;
        for (; i<blocks; i++)
            cipher.ProcessString(buf, BUF_SIZE);
        timeTaken = double(::clock() - start) / CLOCK_TICKS_PER_SECOND;
    }
    while (timeTaken < 2.0/3*timeTotal);

    OutputResultBytes(name, double(blocks) * BUF_SIZE, timeTaken);
}

Prior to calling BenchMark with the StreamTransformation, a factory method created an AES object and keyed it for testing.

Some of the benchmarking code was changed recently, but it was mostly cosmetic. The biggest change was the addition of the random number generators to test how well they perform. Also see RNG Benchmarks and rework of RDRAND/RDSEED on the user list.

When benchmarking, you should also keep in mind the effects of Turbo Boost and other related technologies. If your CPU runs at 3.5 GHz but bursts at 3.8 GHz, then it will affect the expected throughput. At times, it makes the benchmark a moving target, and at other times it may explain why you exceed a theoretical maximum.

The burst is just an internal over-clocking. Intel has been doing it for years. They used to call it NetBurst in the old P5 Pentiums. Hardware guys and gamers have been doing it just as long by using motherboards and processors that support overclocking.

Why AES code in Crypto++ gives different performance results?

2 Answers2