I am trying to calculate an error function with Python 2.7, it has a term such that;
...
sumNSSD = float()
sumNSSD = sumFiGi/sumGi2
return sumNSSD
sumFiGi
and sumGi2
are of type int()
and sumFiGi/sumGi2
results to be something between 0 and 1, lets say 3/4
When the code evaluates 3/4 it returns 0 because of truncation, and sumNSSD
becomes int()
again. When I try sumNSSD = float(sumFiGi/sumGi2)
it still truncates and returns 0.0
A solution that works is to type sumNSSD = float(sumFiGi) / float(sumGi2)
but the actual equation is much longer and it doesn't make sense to put every term inside float(...)
There should be an obvious way for getting around the truncation, however, I did search a lot of topics but no question answers my problem.
Using Decimal()
produces the same result because it evaluates what is in between the parantheses first, then truncates and returns 0
Even Python Documentation puts it this way;
>>> from decimal import * >>> getcontext().prec = 6 >>> Decimal(1) / Decimal(7) Decimal('0.142857')
What would be a simple solution except putting every single term into Decimal(...)
or float(...)