Why floating-points number's significant numbers is 7 or 6

Question

I see this in Wikipedia log 2²⁴ = 7.22.

I have no idea why we should calculate 2^24 and why we should take log10......I really really need your help.

Answer 1

why floating-points number's significant numbers is 7 or 6 (?)

Consider some thoughts employing the Pigeonhole principle:

1. binary32 float can encode about 2³² different numbers exactly. The numbers one can write in text like 42.0, 1.0, 3.1415623... are infinite, even if we restrict ourselves to a range like -10³⁸ ... +10³⁸. Any time code has a textual value like 0.1f, it is encoded to a nearby float, which may not be the exact same text value. The question is: how many digits can we code and still maintain distinctive float?
2. For the various powers-of-2 range, 2²³ (8,388,608) values are normally linearly encoded.
3. Example: In the range [1.0 ... 2.0), 2²³ (8,388,608) values are linearly encoded.
4. In the range [2³³ or 8,589,934,592 ... 2³⁴ or 17,179,869,184), again, 2²³ (8,388,608) values are linearly encoded: 1024.0 apart from each other. In the sub range [9,000,000,000 and 10,000,000,000), there are about 976,562 different values.

Put this together ...

5. As text, the range [1.000_000 ... 2.000_000), using 1 lead digit and 6 trailing ones, there are 1,000,000 different values. Per #3, In the same range, with 8,388,608 different float exist, allowing each textual value to map to a different float. In this range we can use 7 digits.

6. As text, the range [9,000,000 × 10³ and 10,000,000 × 10³), using 1 lead digit and 6 trailing ones, there are 1,000,000 different values. Per #4, In the same range, there are less than 1,000,000 different float values. Thus some decimal textual values will convert to the same float. In this range we can use 6, not 7, digits for distinctive conversions.

The worse case for typical float is 6 significant digits. To find the limit for your float:

#include <float.h>
printf("FLT_DIG = %d\n", FLT_DIG);  // this commonly prints 6

---

... no idea why we should calculate 2^24 and why we should take log10

2²⁴ is a generalization as with common float and its 24 bits of binary precision, that corresponds to fanciful decimal system with 7.22... digits. We take log10 to compare the binary float to decimal text.

2²⁴ == 10^7.22...

Yet we should not take 2²⁴. Let us look into how FLT_DIG is defined from C11dr §5.2.4.2.2 11:

number of decimal digits, q, such that any floating-point number with q decimal digits can be rounded into a floating-point number with p radix b digits and back again without change to the q decimal digits,

p log₁₀ b ............. if b is a power of 10
⎣(p − 1) log₁₀ _b_⎦.. otherwise

Notice "log₁₀ 2²⁴" is same as "24 log₁₀ 2".

As a float, the values are distributed linearly between powers of 2 as shown in #2,3,4.

As text, values are distributed linearly between powers of 10 like a 7 significant digit values of [1.000000 ... 9.999999]*10^{some_exponent}.

The transition of these 2 groups happen at different values. 1,2,4,8,16,32... versus 1,10,100, ... In determining the worst case, we subtract 1 from the 24 bits to account for the mis-alignment.

⎣(p − 1) log₁₀ _b_⎦ --> floor((24 − 1) log10(2)) --> floor(6.923...) --> 6.

Had our float used base 10, 100, or 1000, rather than very common 2, the transition of these 2 groups happen at same values and we would not subtract one.

Answer 2

An IEEE 754 single-precision float has a 24-bit mantissa. This means it has 24 binary bits' worth of precision.

But we might be interested in knowing how many decimal digits worth of precision it has.

One way of computing this is to consider how many 24-bit binary numbers there are. The answer, of course, is 2²⁴. So these binary numbers go from 0 to 16777215.

How many decimal digits is that? Well, log10 gives you the number of decimal digits. log10(2²⁴) is 7.2, or a little more than 7 decimal digits.

And look at that: 16777215 has 8 digits, but the leading digit is just 1, so in fact it's only a little more than 7 digits.

(Of course this doesn't mean we can represent only numbers from 0 to 16777215! It means we can represent numbers from 0 to 16777215 exactly. But we've also got the exponent to play with. We can represent numbers from 0 to 1677721.5 more or less exactly to one place past the decimal, numbers from 0 to 167772.15 more or less exactly to two decimal points, etc. And we can represent numbers from 0 to 167772150, or 0 to 1677721500, but progressively less exactly -- always with ~7 digits' worth of precision, meaning that we start losing precision in the low-order digits to the left of the decimal point.)

The other way of doing this is to note that log10(2) is about 0.3. This means that 1 bit corresponds to about 0.3 decimal digits. So 24 bits corresponds to 24 × 0.3 = 7.2.

(Actually, IEEE 754 single-precision floating point explicitly stores only 23 bits, not 24. But there's an implicit leading 1 bit in there, so we do get the effect of 24 bits.)

Answer 3

Let's start a little smaller. With 10 bits (or 10 base-2 digits), you can represent the numbers 0 upto 1023. So you can represent up to 4 digits for some values, but 3 digits for most others (the ones below 1000).

To find out how many base-10 (decimal) digits can be represented by a bunch of base-2 digits (bits), you can use the log10() of the maximum representable value, i.e. log10(2^10) = log10(2) * 10 = 3.01....

The above means you can represent all 3 digit — or smaller — values and a few 4 digits ones. Well, that is easily verified: 0-999 have at most 3 digits, and 1000-1023 have 4.

Now take 24 bits. In 24 bits you can store log10(2^24) = 24 * log(2) base-10 digits. But because the top bit is always the same, you can in fact only store log10(2^23) = log10(8388608) = 6.92. This means you can represent most 7 digits numbers, but not all. Some of the numbers you can represent faithfully can only have 6 digits.

The truth is a bit more complicated though, because exponents play role too, and some of the many possible larger values can be represented too, so 6.92 may not be the exact value. But it gets close, and can nicely serve as a rule of thumb, and that is why they say that single precision can represent 6 to 7 digits.