12

I've just been testing a program I'm working, and I see that it's executing 3μs faster (a statistically significant change) when I compile it with -g. This makes no sense to me - I thought that the -g flag wasn't supposed to affect the program execution, and that even if it did it would make it run slower, not faster.

Can anyone tell me why this is happening? And whether it changes the programs execution flow? I am not compiling with -O because I need it to execute exactly as written, but if -g can somehow make it run faster with changing the instruction order I should obviously be using that.

So I need to know exactly what changes the -g flag makes to the program.

Edit: The more tests I run, the bigger the t-value gets (= the more statistically significant the difference becomes). This is definitely not measurement error - something is going on.

Benubird
  • 18,551
  • 27
  • 90
  • 141
  • Also: if -g always makes programs run faster, it would be on by default, so what might make my program special? – Benubird Feb 03 '11 at 09:50
  • 5
    Is 3us really a statistically significant change? That's nearing the rate at which the system clock is accurate, and I would be surprised if this anything other than random noise. – templatetypedef Feb 03 '11 at 09:52
  • It is. Over 1000 tests of each version, the t statistic is 1.8 with 40 d.f. That's around 95% significant. – Benubird Feb 03 '11 at 09:55
  • Also: the -g flag changes the standard deviation of test results, which is just weird - I get it might reduce the executing time, but reducing the variance between execution times? – Benubird Feb 03 '11 at 09:57
  • 1
    By 3us you mean 3 microseconds, as in 3μs = 3 * 10^-6 s? I think you can attribute such a small error to measurement error. – R. Martinho Fernandes Feb 03 '11 at 10:19
  • @Martinho - I wish I could, but in this case even a 1μs change will have a substantial impact on the programs effectiveness, nevermind 3μs. If I could run millions of tests to confirm, I would, but I don't have the time. – Benubird Feb 03 '11 at 10:29
  • If you cannot confirm that your measurement error is smaller than 3μs, how can you take any conclusions based on that difference? – R. Martinho Fernandes Feb 03 '11 at 10:42
  • @Martinho systematic measurement error is irrelevant, as I'm comparing the differences, and the values are normally distributed. I'm using welch's t-test with Satterthwaite's approximation, which allows for uncertainty based on random error. Would you reccomend a different test? – Benubird Feb 03 '11 at 11:04
  • 2
    *I am not compiling with -O because I need it to execute exactly as written* -- can you clarify ? Why would you want to throw away a potentially huge performance gain by disabling optimisation and then worry about a 3µs discrepancy ??? – Paul R Feb 03 '11 at 11:46
  • @Paul for business reasons, I need to be able to identify exactly which part of the assembly performs which line of the code, and optimization makes that far harder, so it's not worth it. Speed is not important; I don't care how long it takes, I just need to be able to explain why it takes that long. – Benubird Feb 03 '11 at 12:36
  • possible duplicate [performance hit when using optimization using -g flag][1] [1]: http://stackoverflow.com/questions/89603/how-does-the-debugging-option-g-change-the-binary-executable – gda2004 Sep 23 '14 at 10:39

3 Answers3

10

As others have said, debugging symbols will not change the control flow of your code unless that there is an (unlikely) bug in the compiler.

It changes execution, though, because the executable becomes bigger, and the executed code is spread more widely on more pages. You can expect more cache misses and IO signals. On a multi-tasking environment (and even a Linux/busybox system is such a thing) this can result is slightly different scheduling behavior.

On the other hand, measuring such tiny time differences as you describe them is an art in its own rights. You are probably in an Heisenberg setting, where your measurements influence execution times. Your measurements may show statistically significant deviation, but I would be extremely careful in interpreting them as saying such and such option makes faster code.

Jens Gustedt
  • 76,821
  • 6
  • 102
  • 177
  • That makes sense. But, wouldn't cache misses make the execution slower, not faster? – Benubird Feb 03 '11 at 10:53
  • @Benubird: the perturb the measurements first of all. – Jens Gustedt Feb 03 '11 at 11:05
  • @Jens I'm sorry, I still don't follow. How could cache misses decrease the execution time? – Benubird Feb 03 '11 at 12:37
  • @Benubird, I didn't say that your application itself *is* faster. I said that the combination of application plus your measurements may appear faster. – Jens Gustedt Feb 03 '11 at 12:46
  • @Jens still not understanding how that could happen. Measurements are part of the program, I'm not using something external (like time), measurements are done internally with clock_gettime. How would a cache miss make it appear faster? – Benubird Feb 03 '11 at 13:34
  • @Benubird, you can even have less cache misses, who knows, this depends on the cache strategy of your platform. The executable with the measurements inside is a different one than your production executable, which again is different if you switch debugging on or off. E.g your executable might be slower on startup, large code to load, but then once it is loaded and you start your measurement the cache is still hot. Differences in the range of μs are really difficult to observe reliably. – Jens Gustedt Feb 03 '11 at 14:08
  • 1
    I was about to reply with an "orly?" because debug sections are not loaded and the loaded sections are laid out internally without respect to debug sections and are arranged in the executable file for 1:1 mapping when loaded. However, debug sections seem to prevent unused symbols from being discarded because a debugger might reference them and this affects layout within a section and loaded section sizes. – codeshot Oct 14 '15 at 22:03
8

The -g flag makes 0 changes to the actual generated code. What it does is add debug sections to the executable. Those sections are not loaded at runtime, but debuggers can load them. As the executable now is a bit different it's larger - you might try to measure the no. of page faults going on with one version vs another, there will be changes in how the executable is stored on disk, but no code changes.

If you want to see the assembly, run objdump -d on your binary and compare

I do question the validity of the 3us increase though, reliably measuring 3us, at least on a general purpose OS is a hard task - I hope you have run your program a few thousand times (likely a few hundred thousand times rather) to come up with that number to try to eliminate all the random things affecting such a measurement.

Benjamin Loison
  • 3,782
  • 4
  • 16
  • 33
nos
  • 223,662
  • 58
  • 417
  • 506
  • I'm actually not running it on a general purpose os - it's on a dedicated busybox system, that's only running ptpd, cron, ssh, and this. Timing is pretty accurate, but as execution takes about 1 minute I don't have time to run it hundreds of thousands of times unfortunately. – Benubird Feb 03 '11 at 10:04
1

I am getting a different answer with my code when I use the -debug and -g flags on one of the subroutines, so although I don't know why, yes it is definitely affecting program execution.

Lisa
  • 11
  • 1
  • 1
    This does not provide an answer to the question.To critique or request clarification from an author, leave a comment below their post. – osyan Oct 11 '12 at 12:14
  • 4
    You can't comment if you are below a certain karma threshold (100 I believe) – Michael Feb 13 '13 at 21:13