Using Double for financial Software

Question

I know that this question has already been discussed several times but I am not entirely satisfied with the answer. Please don't respond "Doubles are inaccurate, you can't represent 0.1! You have to use BigDecimal"...

Basically I am doing a financial software and we needed to store a lot of prices in memory. BigDecimal was too big to fit in the cache so we have decided to switch to double. So far we are not experiencing any bug for the good reason and we need an accuracy of 12 digits. The 12 digits estimations is based on the fact that even when we talk in million, we are still able to deal with cents.

A double gives a 15 significant decimal digit precision. If you round your doubles when you have to display/compare them, what can goes wrong??

I guess on problem is the accumulation of the inaccuracy, but how bad is it? How many operations will it take before it affect the 12th digit?

Do you see any other problems with doubles?

EDIT: About long, that's definitely something that we have thinked about. We are doing a lot of division multiplication and long won't deal well with that (losing the decimal and overflow), or at least you have to be very very careful with what you do. My question is more about the theory of doubles, basically how bad is it and is the inaccuracy acceptable?

EDIT2: Don't try to solve my software, I am fine with inaccuracy :). I re-word the question : How likely an inaccuracy will happen if you need only 12digits and that you round doubles when displaying/comparing?

Answer 1

If you absolutely can't use BigDecimal and would prefer not to use doubles, use longs to do fixed-point arithmetic (so each long value would represent the number of cents, for example). This will let you represent 18 significant digits.

I'd say use joda-money, but this uses BigDecimal under the covers.

---

Edit (as the above doesn't really answer the question):

Disclaimer: Please, if accuracy matters to you at all, don't use double to represent money. But it seems the poster doesn't need exact accuracy (this seems to be about a financial pricing model which probably has more than 10**-12 built-in uncertainty), and cares more about performance. Assuming this is the case, using a double is excusable.

In general, a double cannot exactly represent a decimal fraction. So, how inexact is a double? There's no short answer for this.

A double may be able to represent a number well enough that you can read the number into a double, then write it back out again, preserving fifteen decimal digits of precision. But as it's a binary rather than a decimal fraction, it can't be exact - it's the value we wish to represent, plus or minus some error. When many arithmetic operations are performed involving inexact doubles, the amount of this error can build up over time, such that the end product has fewer than fifteen decimal digits of accuracy. How many fewer? That depends.

Consider the following function that takes the nth root of 1000, then multiplies it by itself n times:

private static double errorDemo(int n) {
    double r = Math.pow(1000.0, 1.0/n);
    double result = 1.0;
    for (int i = 0; i < n; i++) {
        result *= r;
    }
    return 1000.0 - result;
}

Results are as follows:

errorDemo(     10) = -7.958078640513122E-13
errorDemo(     31) = 9.094947017729282E-13
errorDemo(    100) = 3.410605131648481E-13
errorDemo(    310) = -1.4210854715202004E-11
errorDemo(   1000) = -1.6370904631912708E-11
errorDemo(   3100) = 1.1107204045401886E-10
errorDemo(  10000) = -1.2255441106390208E-10
errorDemo(  31000) = 1.3799308362649754E-9
errorDemo( 100000) = 4.00075350626139E-9
errorDemo( 310000) = -3.100740286754444E-8
errorDemo(1000000) = -9.706695891509298E-9

Note that the size of the accumulated inaccuracy doesn't increase exactly in proportion to the number of intermediate steps (indeed, it's not monotonically increasing). Given a known series of intermediate operations we can determine the probability distribtion of the inaccuracy; while this will have a wider range the more operations there are, the exact amount will depend on the numbers fed into the calculation. The uncertainty is itself uncertain!

Depending on what kind of calculation you're performing, you may be able to control this error by rounding to whole units/whole cents after intermediate steps. (Consider the case of a bank account holding $100 at 6% annual interest compounded monthly, so 0.5% interest per month. After the third month of interest is credited, do you want the balance to be $101.50 or $101.51?) Having your double stand for the number of fractional units (i.e. cents) rather than the number of whole units would make this easier - but if you're doing that, you may as well just use longs as I suggested above.

Disclaimer, again: The accumulation of floating-point error makes the use of doubles for amounts of money potentially quite messy. Speaking as a Java dev who's had the evils of using double for a decimal representation of anything drummed into him for years, I'd use decimal rather than floating-point arithmetic for any important calculations involving money.

Answer 2

Martin Fowler wrote something on that topic. He suggests a Money class with internal long representation, and a decimal factor. http://martinfowler.com/eaaCatalog/money.html

Answer 3

Without using fixed point (integer) arithmetic you can NOT be sure that your calculations are ALWAYS correct. This is because of the way IEEE 754 floating point representation works, some decimal numbers cannot be represented as finite-length binary fractions. However, ALL fixed point numbers can be expressed as a finite length integer; therefore, they can be stored as exact binary values.

Consider the following:

public static void main(String[] args) {
    double d = 0.1;
    for (int i = 0; i < 1000; i++) {
        d += 0.1;
    }
    System.out.println(d);
}

This prints 100.09999999999859. ANY money implementation using doubles WILL fail.

For a more visual explanation, click the decimal to binary converter and try to convert 0.1 to binary. You end up with 0.00011001100110011001100110011001 (0011 repeating), converting it back to decimal you get 0.0999999998603016138.

Therefore 0.1 == 0.0999999998603016138

---

As a sidenote, BigDecimal is simply a BigInteger with an int decimal location. BigInteger relys on an underlying int[] to hold its digits, therefore offering fixed point precision.

public static void main(String[] args) {
    double d = 0;
    BigDecimal b = new BigDecimal(0);
    for (long i = 0; i < 100000000; i++) {
        d += 0.1;
        b = b.add(new BigDecimal("0.1"));
    }
    System.out.println(d);
    System.out.println(b);
}

Output:
9999999.98112945 (A whole cent is lost after 10^8 additions)
10000000.0

Answer 4

Historically, it was often reasonable to use floating-point types for precise calculations on whole numbers which could get bigger than 2^32, but not bigger than 2^52 [or, on machines with a proper "long double" type, 2^64]. Dividing a 52-bit number by a 32-bit number to yield a 20-bit quotient would require a rather lengthy drawn-out process on the 8088, but the 8087 processor can do it comparatively quickly and easily. Using decimals for financial calculations would have been perfectly reasonable, if all values that needed to be precise were always represented by whole numbers.

Nowadays, computers are much more able to handle larger integer values efficiently, and as a consequence it generally makes more sense to use integers to handle quantities which are going to be represented by whole numbers. Floating-point may seem convenient for things like fractional division, but correct code will have to deal with the effects of rounding things to whole numbers no matter what it does. If three people need to pay for something that costs $100.00, one can't achieve penny-accurate accounting by having everyone pay $33.333333333333; the only way to make things balance will be to have the people pay unequal amounts.

Answer 5

If the size of BigDecimal is too large for your cache, than you should convert amounts to long values when they are written to the cache and convert them back to BigDecimal when they are read. This will give you a smaller memory footprint for your cache and will have accurate calculations in your application.

Even if you are able to represent your inputs to calculations correctly with doubles, that doesn't mean that you will always get accurate results. You can still suffer from cancellation and other things.

If you refuse to use BigDecimal for your application logic, than you will rewrite lots of functionality that BigDecimal already provides.

Answer 6

I am going to answer at question by addressing a different part of the problem. Please accept that I am trying to address the root problem not the state question to the letter. Have you looked at all of the options for reducing memory?

1. For example, how are you caching?
2. Are you using a Fly Weight pattern to reduce storage of duplicate numbers?
3. Have you considered representing common numbers in a certain way?
   Example zero is a constant, ZERO.
4. How about some sort of digit range compression, or hierarchy of digits, for example a hash map by major digits? Store a 32 bit within flag or multiple of some kind
5. Hints at a cool difference approach, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.65.2643
6. Is your run of the mill cache doing something less efficient?
7. Pointers are not free, thought about array groups? depending on your problem.
8. Are you storing objects in the cache as well, they are not small, you can serialize them to structs etc, as well.

Look at the storage problem and stop looking to avoid a potential math issue. Typically there is a lot of excess in Java before you have to worry about digits. Even some you can work around them with the ideas above.

Answer 7

You cannot trust doubles in financial software. They may work great in simple cases, but due to rounding, inaccuracy in presenting certain values etc. you will run into problems.

You have no choice but to use BigDecimal. Otherwise you're saying "I'm writing financial code which almost works. You'll barely notice any discrepancies." and that's not something that'll make you look trustworthy.

Fixed point works in certain cases, but can you be sure that 1 cent accuracy is enough now and in the future?

Answer 8

I hope you have read Joshua Bloch Java Puzzlers Traps Pitfalls. This is what he has said in the puzzle 2: Time for a change.

Binary floating-point is particularly ill-suited to monetary calculations, as it is impossible to represent 0.1— or any other negative power of 10— exactly as a finite-length binary fraction [EJ Item 31].