0

I just read about Denormalized floating point numbers, should i replace all zero literals with almost-zero literal to get better performance.

I am afraid that the evil zero constants in my could pollute my performance. Example:

Program 1:

float a = 0.0f;
Console.WriteLine(a);

Program 2:

float b = 1.401298E-45f;
Console.WriteLine(b);

Shouldn't program 2 be 1.000.000 times faster than program 1 since b can be represented by ieee floating point representation in cannonized form ? whereas program 1 has to act with "zero" which is not directly representable.

If so the whole software development industry is flawed. A simple field declaration:

float c;

Would automatically initialize it to zero, Which would cause the dreaded performance hit.

Avoid the hustle mentioning "Premature Optimization is the..., blablabla". Delayed Knowledge of Compilers Optimization Workings could result in the explosion of a nuclear factory. So i would like to know ahead what i am paying, so that i am safe to ignore optimizing it.

Ps. I don't care if float becomes denormalized by the result of a mathematical operation, i have no control in that, so i don't care.

Proof: x + 0.1f is 10 times faster than x + 0 Why does changing 0.1f to 0 slow down performance by 10x?

Question Synopsis: is 0.0f evil ? So all who used it as a constant are also evil?

Community
  • 1
  • 1
user2186597
  • 297
  • 2
  • 10
  • 2
    The linked question does not state or imply that operations on zero itself are slow (other than possibly division by zero, which generates an exceptional result, namely infinity or NaN). Operations on denormals (subnormals) are a different matter, as they require normalization before entering into a computation, and denormalization when they occur as a result. This can cause overhead on many (but not all) processors. – njuffa Dec 27 '16 at 16:38

1 Answers1

4

There's nothing special about denormals that makes them inherently slower than normalized floating point numbers. In fact, a FP system which only supported denormals would be plenty fast, because it would essentially only be doing integer operations.

The slowness comes from the relative difficulty of certain operations when performed on a mix of normals and denormals. Adding a normal to a denormal is much trickier than adding a normal to a normal, or adding a denormal to a denormal. The machinery of computation is simply more involved, requires more steps. Because most of the time you're only operating on normals, it makes sense to optimize for that common case, and drop into the slower and more generalized normal/denormal implementation only when that doesn't work.

The exception to denormals being unusual, of course, is 0.0, which is a denormal with a zero mantissa. Because 0 is the sort of thing one often finds and does operations on, and because an operation involving a 0 is trivial, those are handled as part of the fast common case.

I think you've misunderstood what's going on in the answer to the question you linked. The 0 isn't by itself making things slow: despite being technically a denormal, operations on it are fast. The denormals in question are the ones stored in the y array after a sufficient number of loop iterations. The advantage of the 0.1 over the 0 is that, in that particular code snippet, it prevents numbers from becoming nonzero denormals, not that it's faster to add 0.1 than 0.0 (it isn't).

Sneftel
  • 40,271
  • 12
  • 71
  • 104
  • 2
    Having worked on the design of floating-point units, it is not so much the additional hardware required for normalizing denormal inputs or denormalizing denormal results that often leads to that additional processing to be removed from the common, fast, path (and handled in a separate slow path), but the negative impact on the latency of the operation. Consequently, on architectures where instruction latency is not a major concern, in particular GPUs (which are designed as throughput machines), denormals are often handled in the common path without overhead. – njuffa Dec 27 '16 at 19:49