3

I was curious about these things:

  1. Does the C language always store the floating-point numbers in the normalized form? (That is, is the normalization always applied?)
  2. Does this also hold true for the results obtained after some arithmetic (addition, multiplication)?
  3. Is it dependent on the language or the hardware - FPU?

It would be really helpful if you can cite any sources. I've looked at the IEEE-754 document, but was not able to find any specific statements regarding implementation.

Edit: Assume the IEEE-754 specification

Harsh Kumar
  • 97
  • 6
  • 2
    Very small numbers are denormalized. Otherwise, there's an [implicit 1 bit that is the MSB of the significand](https://en.wikipedia.org/wiki/Significand), and therefore the number has to be normalized. – user3386109 Nov 20 '21 at 04:34
  • AFAIR that is in the IEEE spec. – Dúthomhas Nov 20 '21 at 04:36
  • The result said to be `normalized`, if it is represented with leading 1 bit, i.e. 1.001(2) x 22. (Similarly when the number 0.000000001101(2) x 23 is normalized, it appears as 1.101(2) x 2-6). Omitting this implied 1 on left extreme gives us the mantissa of float number. A normalized number provides more accuracy than corresponding de-normalized number. The implied most significant bit can be used to represent even more accurate significand (23 + 1 = 24 bits) which is called subnormal representation. The floating point numbers are to be represented in normalized form. – vegan_meat Nov 20 '21 at 04:41
  • @shreydeshwal: “Omitting this implied 1 on left extreme gives us the mantissa of float number”: The preferred term is “significand.” A mantissa is the fraction portion of a logarithm. Significands are linear; mantissas are logarithmic. The significand includes the leading digit. Omitting the 1 gives the bits used for the encoding of the primary significand field in IEEE-754 binary formats. This **encoding** without the leading bit is not the **actual** significand. Also, generally, to cover decimal formats as well as binary, normalized means the leading digit is not 0, not just that it is 1. – Eric Postpischil Nov 20 '21 at 14:17
  • @shreydeshwal: “The implied most significant bit can be used to represent even more accurate significand (23 + 1 = 24 bits) which is called subnormal representation”: That is not what subnormal is. Subnormal means the value is belong the normal range; it is below the number that can be represented with the minimum exponent and the minimum normal significand. Re “The floating point numbers are to be represented in normalized form”: Subnormal numbers are not and cannot be represented in normalized form. – Eric Postpischil Nov 20 '21 at 14:19

2 Answers2

7

Does the C language always store the floating-point numbers in the normalized form?

"It depends." As we'll see, it's more the hardware than the C language that determines this.

If the implementation uses something other than IEEE-754, there's not much we can say.

If the implementation does use IEEE-754, then all numbers are always stored normalized except the ones that aren't, namely the subnormals.

Does this also hold true for the results obtained after some arithmetic (addition, multiplication)?

Yes. (More on this below.)

Is it dependent on the language or the hardware - FPU?

It is typically dependent on the hardware. Most of the time, assuming the target processor supports floating-point at all, a C program is compiled straight to native floating-point instructions, without any language- or compiler-imposed extra processing. (This is by contrast to, for example, Java, which does have a language-imposed floating-point definition, which is implemented in part by the JVM.)

The C standard does have an optional section, "Annex F", which specifies a bunch of specific floating-point behavior, conformant with IEEE-754.

Now, if the C implementation adopts Annex F and is conformant with IEEE-754 (typically because the underlying hardware is, too), the answers to your first two questions become even easier. In IEEE-754 binary arithmetic, with one exception, there are no ambiguities of representation. Every number that can be represented in normalized form has exactly one normalized representation. Every number that cannot be represented in normalized form, but that can be represented as a subnormal, has exactly one subnormal representation. These constraints apply to every IEEE-754 floating-point number, including (naturally enough) the results of other operations.

(The exception, as Eric and Chux have reminded me in a comment, is zero, which IEEE-754 has two of, positive and negative.)

So the answer to "Is the result always normalized?" is "No" (because IEEE-754 most definitely has those subnormals, and of course zero), but if the question is "Does every number have a unique representation?", the answer is mostly "Yes." (Except, again, for zero. Or if you're one of the rare few who is doing something with the IEEE-754-2008 decimal formats, which are much less unique.) See also How to distinguish between 1 and zero floating-point values?

The last question, I suppose, is "How many C implementations adopt Annex F?", or, stated another way, "How many processors comply with IEEE-754?" For the CPU's on general-purpose computers (mainframes and personal computers), as far as I know the answer these days is "All of them". GPU's, on the other hand, are deliberately not quite compatible with IEEE-754 (because they can be even more insanely efficient that way). Microprocessors, for "embedded" work, I'm not so sure about. (Often they don't have viable floating-point at all.)

Steve Summit
  • 45,437
  • 7
  • 70
  • 103
  • This answer appears to assume IEEE-754 binary formats. For the decimal formats, it is not true there are no ambiguities of representation. (Also, in binary, the real number zero has two representations, distinguished at the “floating-point data” level (level 2) in the standard.) – Eric Postpischil Nov 20 '21 at 11:36
  • "Every number that can be represented in normalized form has exactly one normalized representation." --> aside from pesky +0.0 and -0.0. Both have the same _value_, yet 2 forms. IIRC, IEEI-754 considers 0.0 a normal, yet my `isnormal(0.0)` returns 0. Further IEEI-754 _decimal_ types have multiple forms for the many of the same value. – chux - Reinstate Monica Nov 20 '21 at 14:45
  • @EricPostpischil and Chux: Thanks for the reminders. I meant to mention ±0, but I had completely forgotten about decimal. – Steve Summit Nov 20 '21 at 16:47
2

It would be really helpful if you can cite any sources.

C 2018 5.2.4.2.2 3 defines a floating-point representation for a number x as x = sbe Σ1≤=k≤p fkb−k, where s is the sign (±1), b is the base, e is an integer exponent in a range from emin to emax, p is the precision (number of base-b digits), and fk are the base-b digits of the significand.

Then paragraph 4 says:

In addition to normalized floating-point numbers (f1 > 0 if x ≠ 0), floating types may be able to contain other kinds of floating-point numbers, such as subnormal floating-point numbers (x ≠ 0, e = emin, f1 = 0) and unnormalized floating-point numbers (x ≠ 0, e > emin, f1 = 0), and values that are not floating-point numbers, such as infinities and NaNs…

That is it; the C standard is otherwise silent about normalization of floating-point numbers, except that Annex F offers an optional binding to IEC 60559 (effectively IEEE 754). So this answers the questions:

Does the C language always store the floating-point numbers in the normalized form? (That is, is the normalization always applied?)

No.

Does this also hold true for the results obtained after some arithmetic (addition, multiplication)?

No.

Is it dependent on the language or the hardware - FPU?

It is up to each C implementation. C implementations may be influenced by hardware or may adopt software implementations of floating-point arithmetic.

Eric Postpischil
  • 195,579
  • 13
  • 168
  • 312
  • I was actually curious about the IEEE-754 representation (binary) only... however, thank you for another detailed answer. One question regarding the last point. What is usually the case? When I write a C program (or conversion from Cython code, for example) what happens? The machine code does not seem to do any software thing.. – Harsh Kumar Nov 20 '21 at 13:53
  • Also, in IEEE-754 specification https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8766229 section 3.4 c, they say "the leading bit of the significand, d0, is implicitly encoded in the biased exponent E." The figure shows the mantissa to start from d1...dp-1 – Harsh Kumar Nov 20 '21 at 13:59
  • 1
    @HarshKumar: One example is that a C implementation may use a software implementation of floating-point arithmetic when it is targeting hardware without floating-point support. (As I recall, there is some software for that in or associated with the GNU standard C library implementation or GCC or related software.) Another example is a C implementation intended to support C source code written for some old platform different from the current hardware, where that C source code made use of floating-point behaviors different from the current hardware. – Eric Postpischil Nov 20 '21 at 14:14
  • @HarshKumar When C was very, very young, first being implemented by Dennis Ritchie for the PDP-11, you typically had to link against a floating-point library if your program had *any* floating-point code in it. I have a memory that there were *four* different variants: one that emulated all the floating-point operations (even basics like addition and subtraction) in software; one that used the optional hardware floating point unit; one that checked for the floating-point unit and used it if it was there, otherwise fell back to software emulation; and I forget what the fourth one was. – Steve Summit Nov 20 '21 at 14:34
  • In the computer center where I first learned C and Unix, one day we upgraded our PDP-11 to add the floating-point unit, and we were momentarily disappointed when our code did *not* get any faster, until we realized we had been using using the software-only emulation library. As soon as we relinked against one of the variants that used the hardware, *then* our code got nicely faster. – Steve Summit Nov 20 '21 at 14:38
  • And I believe this legacy is responsible for the unfortunate situation we still have today, in which C programmers on Unix are obligated to remember to use an explicit `-lm` to ask for the math library if they're using any floating-point library functions. – Steve Summit Nov 20 '21 at 14:39
  • @SteveSummit: That’s not a Unix characteristic. I think it may be part of the GCC or Binutils configuration. On macOS, which is Unix, you do not need `-lm`. – Eric Postpischil Nov 20 '21 at 15:01
  • Thank you guys for the clarification. I learned a lot about software emulations which I did not know earlier. – Harsh Kumar Nov 20 '21 at 15:33