3

Which gcc compiler options may be safely used for numerical programming?

The easy way to turn on optimizations for gcc is to add -0# to the compiler options. It is tempting to say -O3. However I know that -O3 includes optimization which are non-save in the sense that results of numerical computations may differ once this option is included. Small changes in the result may be insignificant if the algorithm is stable. On the other hand, precision can be an issue for certain math operations, so math optimization can have significant impact.

I find it inconvenient to take compiler dependent issues into account in the process of debugging. I.e. I don't want to wonder whether minor changes in the code will lead to strongly different behavior because the compiler changed its optimizations internally.

Which options are safe to add if I want deterministic--and hence controllable--behavior in my code? Which are almost safe, that is, which options induce only minor uncertainties compared to performance benefits?

I think of options like: -finline -finline-limit=2000 which inlines functions even if they are long.

Philip Conrad
  • 1,451
  • 1
  • 13
  • 22
highsciguy
  • 2,569
  • 3
  • 34
  • 59
  • 2
    If your numerical programs are sensitive enough that floating-point associativity will matter, then you should probably re-examine the algorithms that are being used rather than trying to play with compiler options. – Mysticial Nov 22 '12 at 22:15
  • I tried to emphasize that it is important for me to get identical numerical results independent of internal compiler optimizations which I cannot influence directly. Even for completely stable codes most conceivable math optimizations will lead to tiny differences if they are in place. I don't want this dependence because I want deterministic behavior in the design and debugging stage in which I cannot exclude 'instabilities' of the algorithm. – highsciguy Nov 22 '12 at 22:32
  • 3
    If that's the case, then what you really need is [strict-floating-point](http://stackoverflow.com/questions/7295861/enabling-strict-floating-point-mode-in-gcc). Even the normal optimization levels are not entirely deterministic since the standard allows intermediates to be promoted to higher precision. – Mysticial Nov 22 '12 at 22:35
  • 1
    You should be aware that floating point operations are not the only operations affected by optimizer flags. -O0 may mask missing return statements and many other things may also change, especially if your code is not correct. – Jørgen Fogh Nov 22 '12 at 22:58

1 Answers1

7

It is not true that -O3 includes numerically unsafe optimizations. According to the manual, -O3 includes the following optimization passes in comparison to -O2:

-finline-functions, -funswitch-loops, -fpredictive-commoning, -fgcse-after-reload, -ftree-vectorize and -fipa-cp-clone

You might be referring to -ffast-math, turned on by default with -Ofast, but not with -O3:

-ffast-math Sets -fno-math-errno, -funsafe-math-optimizations, -ffinite-math-only, -fno-rounding-math, -fno-signaling-nans and -fcx-limited-range. This option causes the preprocessor macro __FAST_MATH__ to be defined.

This option is not turned on by any -O option besides -Ofast since it can result in incorrect output for programs that depend on an exact implementation of IEEE or ISO rules/specifications for math functions. It may, however, yield faster code for programs that do not require the guarantees of these specifications.

In other words, all of -O, -O2, and -O3 are safe for numeric programming.

user4815162342
  • 141,790
  • 18
  • 296
  • 355
  • I am mainly using the intel compiler since some time (for related reasons). I remember that gcc didn't realize deterministic behavior in the last version I used for numeric s, because intermediate results were not rounded to machine precision. According to Mystical's link above this shouldn't be the case without fast-math but it definitely was. – highsciguy Nov 22 '12 at 22:47
  • @highsciguy The question does speak of "gcc compiler options", so I assumed gcc. – user4815162342 Nov 22 '12 at 22:50
  • I am talking about gcc (I consider changing back for some applications). I just wanted to mention that I don't have experience with gcc in its most recent versions. – highsciguy Nov 22 '12 at 22:52