0

I tested these calculations

n = 773160767  
n*(n+1)      
n*(n+1)/2
n*(n+1)//2

n*(n+1)/2 % (10**9+7)
n*(n+1)//2 % (10**9+7)

Output:

597777572401189056
2.9888878620059456e+17
298888786200594528

108373072.0
108373040

Resources I find online all use examples like 3/2 where the result is not a whole number. I read https://docs.python.org/3/tutorial/floatingpoint.html too but it also only describes non-whole numbers but my example here uses evenly divisible whole numbers so I expect the result should also be a whole number exactly representable in base2 (298888786200594528) when using float division(/)?

Is there something wrong with the number being too big? If yes, how big is too big and is there a hardware explanation like below from the link above? (I can't do experiments lowering n because I don't know if what I'm seeing is what is represented under the hood)

almost all platforms map Python floats to IEEE-754 “double precision”. 
754 doubles contain 53 bits of precision, 
so on input the computer strives to convert 0.1 to the closest fraction
it can of the form J/2**N where J is an integer containing exactly 53 bits
Han Qi
  • 137
  • 2
  • 7
  • 1
    Does this answer your question? [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) – anthony sottile Nov 10 '20 at 02:16
  • The examples there (and everywhere else I look) seem to use fractions. I'm trying to understand why my division result which will be a whole number (not 0.xxx or requiring anything behind a decimal point) is not returning the correct whole number as given by // – Han Qi Nov 10 '20 at 05:57
  • floats cannot represent all integers – anthony sottile Nov 10 '20 at 05:58
  • @AnthonySottile Thanks! That suggestion led me to a better understanding from https://stackoverflow.com/questions/19473352/are-there-any-whole-numbers-which-the-double-cannot-represent-within-the-min-max, and https://stackoverflow.com/questions/41553774/are-there-number-limitations-in-python/41553937 explained the difference in limits between python integers and floats. – Han Qi Nov 10 '20 at 06:11

0 Answers0