0

When I run:

import numpy as np
print(hex(int(np.float128(2**64-1))))
print(hex(int(np.float128(2**65-1))))

I get:

0xffffffffffffffff
0x20000000000000000

2**64-1 is correctly returned. However, 2**65-1 is not correctly returned. I should have 112 significant bits (mantissa), so this shouldn't be a problem. But it is.

Any idea what's going on here?

philn
  • 654
  • 6
  • 17
Ryan Johnson
  • 102
  • 7
  • floating point imprecision? – SuperStormer Jul 26 '21 at 19:05
  • 3
    `np.float128(2**65-1) == np.float128(2**65)` is `True` when I test it, so the problem is not the `int` conversion, it's the `float128` conversion. – kaya3 Jul 26 '21 at 19:09
  • Basically, `np.float128` is not necessary a quadruple precision binary floating point number. See the linked duplicate candidate (which may just be a duplicate appropriate for closing). – juanpa.arrivillaga Jul 26 '21 at 19:13
  • 4
    ["In spite of the names, np.float96 and np.float128 provide only as much precision as np.longdouble, that is, 80 bits on most x86 machines and 64 bits in standard Windows builds."](https://numpy.org/doc/stable/user/basics.types.html) – chepner Jul 26 '21 at 19:13

0 Answers0