when I run the program for the second time it doesn't give me correct execution time

Question

I have a C code, When I try to calculate the time of small pieces of processing code for the first execution. It gives me 30 ms, when I turn of the exe file and run it again it gives me 1 ms, and this time is the time for calculation and each time I run the program the calculation values is different from the previous one, if I turn off the PC and turn it, it gives me 30 ms for the first execution and 1 ms for all other executions How can I get the same time , I free all the used memory and I run another program to overwrite the memory but the problem is not solved until I reboot the PC any help

start_time=clock();
Encryption();
end_time=clock();
cpu_time_used_totlal_enc +=(double) (end_time-start_time) / CLOCKS_PER_SEC;

The code should be a small, compilable example demonstrating your problem. Please add it to your original post, as it's hard to read as a comment. — nvoigt, May 15 '13 at 14:59
the encryption function called many other functions (20 function) — Mousa Farajallah, May 15 '13 at 15:01
Your computer learns the result on the first run. On second (and later) runs, it just outputs the pre-calculated results. — pmg, May 15 '13 at 15:03
Your first step then should be to reduce your scope and see if the problem persists. If it does, you have a smaller example, if it does not, you just found your culprit. — nvoigt, May 15 '13 at 15:04
no, I read and write to the memory out of this functions, but I have a large number of dynamic arrays, and one of them a bout 123456 values — Mousa Farajallah, May 15 '13 at 15:04
but the result and the values are changing each time I execute the program, I means the values of the first execution are different from those in the second execution and so on — Mousa Farajallah, May 15 '13 at 15:06
before encryption and after it I read and write to the file but inside it no access to the file or memory, just access to the dynamic arrays — Mousa Farajallah, May 15 '13 at 15:07

score 6 · Accepted Answer · answered May 15 '13 at 15:10

6

This problem is called "warmup": When you want to do performance test some code, you need to run the code several times (say, 10 times). Then you run it 100'000 times and measure how long it takes and divide that by 100'000 to get the average. A single measurement of the runtime is useless, unless the runtime is at least one minute.

The reason for warmup problems is that modern OSs and languages do all kind of tricks to make your code execute faster. For example, the call to Encryption() might actually invoke a function in a shared library.

Those libraries are loaded lazily, i.e. it's loaded the first time when your code actually calls the function. When it is loaded, the OS keeps it in a cache since chances are someone will need it again.

That's why the first few runs of an application have a completely different runtime than the next 10'000 runs.

answered May 15 '13 at 15:10

Aaron Digulla

321,842
108
597
820

if I run my program 1000 times after the first execution it gives me the same time (1 ms) while in the first time it gives me 30 ms, I think the first execution is the real performance – Mousa Farajallah May 15 '13 at 15:13
@MousaFarajallah: As I said, this assumption is completely wrong. It takes 30ms to fill all the caches and maybe prepare the sources of randomness of the OS. The real runtime is 1ms. Or do you think a program could run faster than it "really" can run? What do you think happens when it only takes 1ms? Do you think the OS is just "faking" to run it? – Aaron Digulla May 15 '13 at 15:20
I do not know, but when I increase the size of the image under test from 49128 to 786432 it gives me almost the same time for the first and other executions – Mousa Farajallah May 15 '13 at 15:36
how can I force the program to not do the other operations like fill all the caches and other operations, I mean how can I get the correct performance time, I think you are write and I agree with you – Mousa Farajallah May 15 '13 at 15:38
I mean the company need the program to run it one time, so it assumes my program is slow – Mousa Farajallah May 15 '13 at 16:08
1

@MousaFarajallah, Aaron is right. I think the warmup time is most probably caused by disk I/O. Since the first time you run your program it is not stored on the OS disk cache, it needs to access the hard disk in order to load the executable along with all its dependencies. If you disable the cache, you'll probably find that the execution time becomes consistent, but for the worse (~30 ms each time it is run). Anyways, 30 ms is lightning fast if the company expects to run the program just every once in a while. =) – May 15 '13 at 18:54
@MousaFarajallah: If the company needs the program to be fast, they can run it a few times to prefill caches before they actually need it. One way to do this would be to start it in a cron job a few minutes before the business day starts, for example. Also make sure that you configured your random sources correctly; `/dev/random` on Linux can be very slow: http://stackoverflow.com/questions/4819359/dev-random-extremely-slow – Aaron Digulla May 16 '13 at 07:52
Ok, But this program is to encrypt images, for example if we run AES one or 2 or 100 times the time is the same, and I think AES is not stored on the OS disk cache, while in my program it needs that, how can I speed it, since the company to accept my program consider the first execution time is the practical time, any help please – Mousa Farajallah May 22 '13 at 16:15
How to disable the cache, I used two techniques but the problem of time difference between the first and the second execution time still found – Mousa Farajallah May 22 '13 at 16:19
To disable the caches, download the sources for your OS and remove all the caching code. Then buy a CPU from ca. 1990 - they don't have many caches. Mainboards from that time can be found in many electronic dumps in Africa, for example. Shouldn't take more than 2-3 years to do it. More seriously, anyone with a little bit of experience in performance knows "warm up" and what it means. If the company doesn't understand the technology they use, they have to trust you. If they don't then there is no basis to do business with them. – Aaron Digulla May 23 '13 at 07:09
Note for Java code, the warm up is 1'500 to 10'000 executions of a method to make sure the VM has optimized the code. http://stackoverflow.com/questions/1481853/technique-or-utility-to-minimize-java-warm-up-time – Aaron Digulla May 23 '13 at 07:11

when I run the program for the second time it doesn't give me correct execution time

1 Answers1