The problem is that for both float and double, you are using 32 and 64 bits respectively for representation of both the integer and decimal part of the number. The problem comes in when you try to represent a fractional value which doesn't really have an accurate decimal representation in binary bits.
Take .1 for example, there's no way to exactly represent this in base 2, any more than there is a way to accurately represent 1/3 in base 10.
So java uses some tricks so that when you say:
float f = 3.1;
System.out.println(f);
It prints out the right number. However, when you start doing arithmetic with these values, you end up with rounding errors.
BigDecimal is accurate because it uses a different representation. It internally stores a BigInteger (which uses an int[] to represent huge numbers). It then uses a precision value to tell it how many of those integer digits are are after the decimal point.
For instance, the value 3.1 would be represented in a BigDecimal as 31,precision=1
For this reason, BigDecimal doesn't suffer from the same rounding issues a float and double.
However, when you use a float/double value to initialize a BigDecimal, the same rounding error makes it's way into the BigDecimal instance. That's why it's recommended to use a String to construct the value.