0

I've been using the following way to format decimals to strings:

NSString(format: "%\(0.2)f", DECIMAL)

However I noticed something odd which is that 1.025 rounds to 1.02 where as 1.026 rounds to 1.03. Is this correct behavior?

I remember in math we always would round up at .5. Is this an expected behavior I've never noticed? Or is there a better way to do this.

enter image description here

NIST rounding rules are as such: (http://physics.nist.gov/Pubs/SP811/appenB.html#B.7.2)

B.7.2 Rounding converted numerical values of quantities

The use of the factors given in Secs. B.8 and B.9 to convert values of quantities was demonstrated in Sec. B.3. In most cases the product of the unconverted numerical value and the factor will be a numerical value with a number of digits that exceeds the number of significant digits (see Sec. 7.9) of the unconverted numerical value. Proper conversion procedure requires rounding this converted numerical value to the number of significant digits that is consistent with the maximum possible rounding error of the unconverted numerical value.

Example: To express the value l = 36 ftin meters, use the factor 3.048 E-01 from Sec. B.8 or Sec. B.9 and write

l = 36 ft × 0.3048 m/ft = 10.9728 m = 11.0 m. The final result, l = 11.0 m, is based on the following reasoning: The numerical value "36" has two significant digits, and thus a relative maximum possible rounding error (abbreviated RE in this Guide for simplicity) of ± 0.5/36 = ± 1.4 % because it could have resulted from rounding the number 35.5, 36.5, or any number between 35.5 and 36.5. To be consistent with this RE, the converted numerical value "10.9728" is rounded to 11.0 or three significant digits because the number 11.0 has an RE of ± 0.05/11.0 = ± 0.45 %. Although this ± 0.45 % RE is one-third of the ± 1.4 % RE of the unconverted numerical value "36," if the converted numerical value "10.9728" had been rounded to 11 or two significant digits, information contained in the unconverted numerical value "36" would have been lost. This is because the RE of the numerical value "11" is ± 0.5/11 = ± 4.5 %, which is three times the ± 1.4 % RE of the unconverted numerical value "36." This example therefore shows that when selecting the number of digits to retain in the numerical value of a converted quantity, one must often choose between discarding information or providing unwarranted information. Consideration of the end use of the converted value can often help one decide which choice to make.

Note: Consider that one had been told initially that the value l = 36 ft had been rounded to the nearest inch. Then in this case, since l is known to within ± 1 in, the RE of the numerical value "36" is ± 1 in/(36 ft × 12 in/ft) = ± 0.23 %. Although this is less than the ± 0.45 % RE of the number 11.0, it is comparable to it. Therefore, the result l = 11.0 m is still given as the converted value. (Note that the numerical value "10.97" would give excessive unwarranted information because it has an RE that is one-fifth of ± 0.23 %.)

zrzka
  • 20,249
  • 5
  • 47
  • 73
Jeef
  • 26,861
  • 21
  • 78
  • 156
  • The format should be `"%0.2f"`, without the string interpolation. – Martin R Jul 08 '15 at 15:06
  • The real problem is that the number 1.025 cannot be represented exactly as a binary floating point number, so `a` might be slightly less than 1.025. I am sure that this has been answered before somewhere. – Martin R Jul 08 '15 at 15:06
  • Your examples don't match your screenshot, and they don't help making the question very clear... – Eric Aya Jul 08 '15 at 15:07
  • @MartinR yep, check with [IEEE 754 Converter](http://www.h-schmidt.net/FloatConverter/IEEE754.html). 1.025 is actually 1.024999976158142 thus it's rounded to 1.02 even if format is set correctly to "%0.2f". – zrzka Jul 08 '15 at 15:09
  • @Jeef better use [NSNumberFormatter](https://developer.apple.com/library/prerelease/ios/documentation/Cocoa/Reference/Foundation/Classes/NSNumberFormatter_Class/index.html#//apple_ref/doc/uid/20000202-SW16) where you can configure [Rounding Behavior](https://developer.apple.com/library/prerelease/ios/documentation/Cocoa/Reference/Foundation/Classes/NSNumberFormatter_Class/index.html#//apple_ref/doc/uid/20000202-SW16). – zrzka Jul 08 '15 at 15:10
  • Just FYI - [What Every Computer Scientist Should Know About Floating-Point Arithmetic](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) – zrzka Jul 08 '15 at 15:18

0 Answers0