3

This is not a question about how floating point works. Nor is it a question about why 19.95 * 0.1 produces 19.900000000004. I already get that floating point math isn't a perfect presentation. The question is rather to ask an expert their intuition why this particular case works.

Recently I wanted to round numbers to an arbitrary amount for a stepping slider. I had this code

 steppedV = Math.round(v / step) * step

I found it's not very good because of floating point math issues. For example v = 19.94 step = 0.1 returns 19.900000000000002

I thought I was kind of out of luck but then just for fun I tried this

 steppedV = Math.round(v / step) / (1 / step)

And surprisingly, for the test cases I've been using it's producing better results. For the same inputs as above I get back 19.9

For all my cases that were not rounding to my test step values it's all working now. Can someone who really groks how floating point math works explain why / (1 / step) produces more accurate results and * step does not? As someone who understands floating point math is this result intuitive? Am I just getting lucky or as an expert would you have known to do the division of (1 / step) for a better result?

function test(start, end, inc, step) {
  const addRow = makeTable(step, 'div', 'mult');
  for (let v = start; v <= end; v += inc) {
    const steppedVMult = Math.round(v / step) * step;
    const steppedVDiv = Math.round(v / step) / (1 / step);
    addRow(v, steppedVDiv, steppedVMult);
  }
}

test(0, 2, 0.03, 0.1);
test(0, 0.2, 0.003, 0.01);
test(0, 0.02, 0.0003, 0.001);
test(0, 0.002, 0.00003, 0.0001);

function c(tag, children = [], text = '') {
  const e = document.createElement(tag);
  e.textContent = text;
  children.forEach(c => e.appendChild(c));
  return e;
}
  
function makeTable(...args) {
  const makeRow = arr => c('tr', arr.map(v => c('td', [], v)));

  const tbody = c('tbody');
  const table = c('table', [
    c('thead', [makeRow(args)]),
    tbody,
  ]);
  document.body.appendChild(table);
  return function(...args) {
    tbody.appendChild(makeRow(args));
  };
}
body { font-family: monospace; }
table { border-collapse: collapse; margin-bottom: 10px }
thead { font-weight: bold; }
td { border: 1px solid #888; padding: 3px; }
ChrisGPT was on strike
  • 127,765
  • 105
  • 273
  • 257
samanthaj
  • 521
  • 3
  • 14
  • 19.900000000000002 is *extremely* close to 19.9 – Pointy Jul 06 '22 at 03:00
  • that's not the point – samanthaj Jul 06 '22 at 03:00
  • Binary floating point values have to be converted to decimal notation *for display*. That process makes decisions about how many decimal places to show. Decimal floating point and binary floating point are not compatible, truly, so there will very very frequently be apparent inconsistency. – Pointy Jul 06 '22 at 03:05
  • Also, your workaround may produce its own unexpected results in other circumstances. – Pointy Jul 06 '22 at 03:06
  • You're right, that's not the point, it's pointy ... anyway, read https://stackoverflow.com/questions/588004/is-floating-point-math-broken – Jaromanda X Jul 06 '22 at 03:08
  • sigh...., @JaromandaX, that wasn't the point of the question. I can read every part of the answer you linked. My question is not "how does floating point work". My question is why does this particular case work. The link doesn't lead to an answer, it leads to too much info. you're basically asking me to work it out from first principles instead of being helpful. – samanthaj Jul 06 '22 at 03:14
  • @samanthaj - I think that question is highly educational in understanding floating point limitations ... sorry to have annoyed you – Jaromanda X Jul 06 '22 at 03:17
  • The way to get the answer you seek is to look at the floating-point values *in binary*. You can write some code (in JavaScript) to do that; it's kind-of a pain, but it may illustrate the difference between the two computed values. – Pointy Jul 06 '22 at 03:17
  • You're missing the point, @Pointy. From my POV I asked for an expert to share their intuition. Instead it's been closed with a "read the details of the hardware" and figure it out yourself. Of course that's always possible and that could be the answer to every question on this site "Go read the manual and GTFO" but the point of asking a question like this is for an expert to share their insight, not to just say "go read the manual". – samanthaj Jul 06 '22 at 03:21
  • The easy way to look at the raw binary is to make a `Float64Array` with one element. Put a value in that element. Then take the backing buffer from that array and make a `UInt8Array` with 8 elements from that. Now you have 8 integer values that you can display as a string of binary digits with some simple code. – Pointy Jul 06 '22 at 03:22
  • You're asking for "insight", but that's not what's required here. What you're asking is for somebody to figure out exactly what your two different values look like in their binary representations. If I had a tool to do that, I could, but I don't. I doubt there are many people around who could simply look at numbers like those in your question and tell you exactly why you got the results you did. – Pointy Jul 06 '22 at 03:24
  • @Pointy, That won't tell me why division productions the bits that make 19.95 and multiplication make the bits that product 19.9500000000004 – samanthaj Jul 06 '22 at 03:25
  • It might if you looked at all the intermediate results. Even the translation of a decimal-notation number into binary may be "off", so every expression will have its own unique behavior. – Pointy Jul 06 '22 at 03:26
  • I think following the logic displayed by Pointy and JaromandaX here pretty much all questions on S.O. should be closed. Want to know why 3D Matrix math works? Closed! Write out all the equations and figure it out on your own! There are lots of math "tricks" in computing that are well known by experts. I suspect someone out there knows "Oh, yea, I do this all the time because ...." – samanthaj Jul 06 '22 at 03:30
  • 1
    `0.1` cannot be correctly represented in floating point. It's exact value is `0.1000000000000000055511151231257827021181583404541015625`. When you multiply by this number at some point the trailing values (..5551115...etc.) will get involved. However the exact value of `1/0.1000000000000000055511151231257827021181583404541015625` cannot fit into 64 bit floats therefore the result is `10.000000000000000000000000000000000000000000000000` or exactly `10` not due to rounding but due to the fact that the trailing decimals cannot fit into a 64 bit float format. – slebetman Jul 06 '22 at 05:33
  • FYI javascript provides a function to check the exact floating point value (represented in decimal) of a floating point number: `number.toPrecision(some_big_number)`. For example `(0.3).toPrecision(60)` is `0.299999999999999988897769753748434595763683319091796875000000` and `(0.25).toPrecision(60)` is `0.250000000000000000000000000000000000000000000000000000000000`. If you remove the trailing `0`s and the number is the same as your input then it means that your input number can be exactly represented in 64 bit floating point format – slebetman Jul 06 '22 at 05:39
  • ... Heck, you can automate that with `number.toPrecision(100).replace(/0*$/,'')` – slebetman Jul 06 '22 at 05:41
  • ... Notice that maybe this works out for `0.1` because of your luck. If the step value is a different number you may not be so lucky to have the trailing decimals conveniently not fit into 64 bit floating point format. – slebetman Jul 06 '22 at 05:43
  • Voting to reopen - there's plenty to explain here beyond "is floating-point math broken". – Mark Dickinson Jul 06 '22 at 07:56
  • @Pointy: Nothing in the question asks to figure out the binary representation as would be shown in the backing buffer. That would be the encoding for the value, which is irrelevant to the question. The necessary information would be the floating-point representation, not its encoding as a bit string. The floating-point representation is the representation of a number as a signed significand multiplied by the base to the power of an exponent. It is a mathematical question. – Eric Postpischil Jul 06 '22 at 10:03
  • I am tempted to vote to re-open so I can answer, but a proper answer would take some detailed analysis of the cases, which I am not going to devote time to at the moment. Since the question asks for intuition, I will say this: How the result is rounded in each floating-point operation is dictated by mathematics; it is deterministic and immutable. However, for practical purposes, it often behaves as if it is random. Some operations will round up on particular operands, some will round down, and a few may be exact… – Eric Postpischil Jul 06 '22 at 10:05
  • … Since looking into the mathematical details of each operation is hard, we do not know what the result will be for each operation until we look at it in detail. When we are not looking in details, it is “random in practice”: Rounding up or down occurs in ways we do not predict. Sometimes these roundings happen to reinforce; a division by a particular q may round down, and that produces a new number such that a subsequent multiplication by the same q may happen to round down also, so the result is lower than it would be in real-number mathematics… – Eric Postpischil Jul 06 '22 at 10:08
  • … Sometimes these roundings happen to cancel; a division may round down, and the subsequent multiplication may happen to round up. My suspicion is that, in the cases you have discovered, the roundings in the `1 / step` merely happen to have canceled (or otherwise behaved nicely for your purpose) due to these essentially arbitrary effects. There is likely nothing remarkable about it, and it probably will not persist with other values of `step` and `v`. – Eric Postpischil Jul 06 '22 at 10:10
  • It should be said again that the roundings are not truly random, and there are patterns in them, and sometimes these account for multiple examples occurring in a pattern, such as the rounding behavior nicely for all of 2, .2, .02, and .002, but possibly failing for .07. I answered some other Stack Overflow question in the past where the behavior depended on the bits that happened to appear in one number involved, say 11.01100001 in binary, and the behavior persisted where there were zero bits in that (brought into play by various scalings elsewhere) but stopped when it reached the ones. – Eric Postpischil Jul 06 '22 at 10:13
  • @EricPostpischil I was in fact talking about examining the actual bit pattern of the 64-bit representation. That's what the "buffer" is: not a string, but a set of bytes. By using the same buffer for a UInt8Array, it's easier to "take apart" the exponent and mantissa because bytes are easy to deal with. – Pointy Jul 06 '22 at 11:45
  • By doing that, it would be clear how the *actual* values of the two different expressions in the OP differ, on a bit-by-bit level. – Pointy Jul 06 '22 at 11:46
  • @Pointy: And that bit pattern is what I was saying is irrelevant. It is a bit string, per the IEEE-754 standard; the grouping into bytes is just cosmetic. – Eric Postpischil Jul 06 '22 at 12:18
  • @EricPostpischil OK I am not making myself clear. Using unsigned bytes (which, when accessed, become plain numbers) makes it easy to use shifting and masking to *then* produce a presentation of the exponent (bits) and mantissa (bits). A fancier version could highlight the differing bit positions between two numbers. Now, as you say, those are not interpretable by themselves, but in a *comparison* between two particular expressions that one might think should be the same (as in the OP), at least the actual difference would be apparent. – Pointy Jul 06 '22 at 12:42
  • @Pointy: There is no need to look at the bytes representing a `Number` to see its value or differences between Number values. To see the exact value of a `Number` `x`, simply display `x.toString(16)`. This will convert it to hexadecimal (even `Number` values that are not integers) and, because the base 16 used for hexadecimal is a power of the two used for the floating-point base, there will be no rounding errors. The conversion will be exact. – Eric Postpischil Jul 06 '22 at 13:10
  • @EricPostpischil that produces a strange representation of the number. Looking at the actual base-2 exponent and actual mantissa is (and has often been, in my experience) much more useful. – Pointy Jul 06 '22 at 13:48

0 Answers0