When performing arithmetics code gives unusual results

Question

Python script seems to do incorrect arithmetics.

import _thread
import time

def mult(pixel : int, multiplier : tuple) -> float:
    "multiplies either r,g,b pixel by row vector (0.257, 0.504, 0.098),(-0.148,-0.291,0.439),(0.439,-0.368,-0.071))"

    sum=0
    for mul in multiplier:
        sum += (pixel * mul)
        print(sum)
    return sum

redMul = [0.257, 0.504,0.098]
greenMul = [-0.148, -0.291, 0.439]
blueMul = [0.439, -0.368,-0.071]

redAdd = 16
greenAdd = 128
blueAdd = 128

try:
   _thread.start_new_thread(mult,(1,blueMul))
except:
    print("Error: unable to start thread")

code outputs:

0.439
0.07100000000000001
1.3877787807814457e-17

when expecting:

0.439 
0.071
0

What is the cause of this error and how can i resolve it?

Floating point math: https://docs.python.org/3/tutorial/floatingpoint.html — user812786, Mar 17 '16 at 15:41
Adjust your expectations or use the function `round()` on the final results if you only want a few decimal places of accuracy. — John Coleman, Mar 17 '16 at 15:44

Haifeng Zhang · Answer 1 · 2016-03-17T16:14:58.943

1

sum = round(sum, 3))

limit float to 3 decimal points, and then do the arithmetic

def mult(pixel : int, multiplier : tuple) -> float:
    "multiplies either r,g,b pixel by row vector (0.257, 0.504, 0.098),(-0.148,-0.291,0.439),(0.439,-0.368,-0.071))"

    sum=0
    for mul in multiplier:
        sum += (pixel * mul)
        sum = round(sum, 3)
        print(sum)
    return sum

edited Mar 17 '16 at 16:14

answered Mar 17 '16 at 15:50

Haifeng Zhang

30,077
19
81
125

round(sum,3) rounds sum to 3dp, what does the rest of your solution add? – dxpelou Mar 17 '16 at 16:11
im refering to the " float("{0:.3f}".format()) " – dxpelou Mar 17 '16 at 16:13
@LouAnimashaun I modified the code. that's for string formatting. it is useless in this case – Haifeng Zhang Mar 17 '16 at 16:18

score 1 · Accepted Answer · answered Mar 17 '16 at 16:11

The Python script does not do incorrect arithmetic, it is just the way floating-point arithmetic is.

The main principle you need to be aware of when using float (double floating point precision in Python) is that it is an approximation of the actual value. Unless the value is having denominator of power 2 - like 1, 2, 4, 0.5, 0.25, etc, the value cannot be exactly represented by float. It is always approximation, up to the 15/16-th digit.

Thus all your results:

0.439
0.07100000000000001
1.3877787807814457e-17

Are correct... up to the 15/16th significant digit, that is:

0.439
0.0710000000000000
0.000000000000000 #(thus you have 1 e-17 as your 17-th digit)

float does not guarantee precision beyond the 15-th/16-th digit

If you really need higher precision than 15-16-th digit, consider using decimal

When performing arithmetics code gives unusual results

2 Answers2