0

In the single precision floating point representation of numbers according to IEEE 754 standard, we use 24 bits for mantissa part (23 bits + 1 implied bit). So the precision can be calculated as 2^24 = 10^x where x can be calculated by taking log on both sides as 24log 2 = xlog 10 => x= 7.2 ~ 7. From this we can conclude that precision is 7 in decimal system but the value 7 tells that we have 7 significant decimal places of precision or we have 7 decimal digits of precision? Is it considered to be 7 decimal digits (in total) or upto 7 decimal places of precision for decimal numbers.

Mark Dickinson
  • 29,088
  • 9
  • 83
  • 120
Harshit Singh
  • 25
  • 5
  • 3
    [It is not good to think of binary floating-point formats in decimal terms.](https://stackoverflow.com/a/61614323/298225) Before you can conclude “the precision is 7,” you would have to define what “the precision” means. There are different ways of defining precision. The IEEE-754 “single” format is sufficiently precise that when any **six**-digit decimal floating-point number within range is converted to the binary floating-point format and back to a six-digit decimal floating-point number, the result equals the starting number. There are seven-digit numbers for which this is not true. – Eric Postpischil Aug 01 '21 at 00:58
  • The IEEE-754 “single” format certainly does not guarantee seven decimal places of precision, meaning seven digits after the decimal point. For numbers above 2^24, only integers are representable; the IEEE-754 “single” values do not have any digits after the decimal point at all. – Eric Postpischil Aug 01 '21 at 01:01

1 Answers1

1

precision can be calculated as 2^24 = 10^x ...

From this we can conclude that precision is 7 in decimal system but the value 7 tells that we have 7 significant decimal places of precision or we have 7 decimal digits of precision?

Is it considered to be 7 decimal digits (in total) or upto 7 decimal places of precision for decimal numbers.

No.

7 may work as a rough approximation of precision*1, yet floating point numbers are not quite distributed in a uniform logarithm fashion so “by taking log” fails.

For typical float, there are 223 or 8,388,608 values, linearly distributed in the range [1.0 … 2.0). Likewise for ranges [0.125 … 0.5), [128.0 … 256.0), …

With text as a decimal with precision of 7, there are 9*106 or 9,000,000 values linearly distributed in the range [1.0 … 10.0) and in other decades.

The problem is these ranges do not always line up to distinguish 7 significant decimal places.

Consider 8589952000.0f. The next float is 1024.0 away, yet the next 7 significant decimal is 1000.0 away. After 125 steps to the next float, the value is 8590080000.0f. But that was 128 steps of 1000.0. Some 7 significant decimals in that range were not distinctively representable as a float.

Some float have less than 7 significant decimals of distinctiveness.

Use floor((24-1) * log10(2)) --> 6 to determine a common float worst-case decimal precision.

*1 log10(pow(2,23)) --> 6.9... is a better approximation.

chux - Reinstate Monica
  • 143,097
  • 13
  • 135
  • 256