6

Whenever I search for the term 'denormal numbers' or 'denormals', I only find ways how to detect them and round them to zero. Apparently, nobody really likes them, because dealing with them incurs a performance penalty.

And yet, they're implemented everywhere. Why? If it's for precision, I'd say you're gonna need a bigger float, or change the order of your operations such that you avoid really small intermediate values. I find it hard to believe that that little bit of extra precision is really worth the precious clock cycles.

Are there any good reasons why one would still use denormal numbers? And if there are no significant reasons to have denormal numbers, why implement them at all? Only to have IEEE754 compliance?)

  • 1
    For reference, here's an example of how bad denormals can be: http://stackoverflow.com/questions/9314534/why-does-changing-0-1f-to-0-slow-down-performance-by-10x – Mysticial May 15 '12 at 16:04

2 Answers2

6

In short, because gradual underflow retains some useful mathematical identities (e.g. x-y == 0 means that x == y). Some explanations of why gradual underflow can be useful:

http://grouper.ieee.org/groups/754/faq.html#underflow

http://www.cs.berkeley.edu/~wkahan/ARITH_17U.pdf

And yes, in some cases underflows are encountered due to bad application design, and the proper action is to fix the application. In other cases applications that work correctly with gradual underflow would fail under abrupt underflow.

Furthermore,

  • In many cases, slow but correct is considered a better default than fast but dangerous.

  • Since gradual underflow is the default, google finds people complaining about that and wanting to turn it off. If OTOH abrupt underflow were the default, maybe instead you'd see more people complaining about mysterious numerical issues? Numerical programming is hard enough as it is!

  • Modern hw has reduced the penalty for working with subnormal numbers. See e.g. http://www.agner.org/optimize/blog/read.php?i=142&v=t

janneb
  • 36,249
  • 2
  • 81
  • 97
  • It looks like you meant 'abrupt' where you wrote 'gradual' at the end of your second paragraph. – Phil Miller May 14 '12 at 13:02
  • "x-y == 0 means that x == y" Please explain [this](http://stackoverflow.com/questions/10334688/how-dangerous-is-it-to-compare-floating-point-values) then. –  May 14 '12 at 13:05
  • 1
    @Tinctorius: absolutely nothing on the page you linked to contradicts that. More to the point, it is a basic *theorem* of floating-point error analysis (i.e. it is mathematically true); there is nothing to "explain". – Stephen Canon May 14 '12 at 13:12
  • I should have been more clear about it. What I meant was: please explain how that is a useful property, in the light of `x==y` or `x-y==0` being bad smells when doing FP calculations. You always need to know your margin of error (and don't forget error-on-error, or whatever it's called again), and comparisons should always be like `|x-y| –  May 14 '12 at 13:15
  • 2
    @Tinctorius: A "code smell" is an overly-coarse guideline for people who don't actually know what they're doing (or can't be bothered to carefully read the sources in question). Exact FP comparisons (especially with zero) are both appropriate and correct in many circumstances; and are absolutely used by those who understand floating-point. – Stephen Canon May 14 '12 at 13:20
  • An example would be nice. I really can't find any, except for very, very safe numbers around one. –  May 14 '12 at 13:30
3

Denormals are extremely useful; there are a number of useful error bounds on floating-point computation which are no longer true if you remove denormals (the most important being that x-y == 0 if and only if x == y).

It's also important to remember that (a) denormals do not incur a penalty on all hardware; there exist systems which can handle denormals at speed (or very close to it) and (b) denormals only slow down your computation if you actually encounter one. If you don't use them, you don't pay for them (and if you do end up using them, then your result would have likely been wrong without them--if you simply want a wrong answer as quickly as possible, you can replace your entire computation with return 0).

Stephen Canon
  • 103,815
  • 19
  • 183
  • 269
  • 1
    If you want a completely correct computation - use bignums. The reason you chose floating point, is to trade precision for efficiency. The same applies for choosing not to use denormals. Indeed you can instruct the compiler to disable them. – Elazar Leibovich Jun 09 '14 at 10:43