% expr 60.9-59.7
1.1999999999999957
It seems that in the above example expr
outputs the result of the floating point operation in higher precision that it computes.
For something that simple I would expect a more reasonable output, such as output by this C++ code:
#include <iostream>
int main()
{
double a = 60.9;
a -= 59.7;
std::cout << "double a = " << a << std::endl;
float b = 60.9;
b -= 59.7;
std::cout << "float b = " << b << std::endl;
}
Output:
double a = 1.2
float b = 1.2
Thus the questions:
Why does TCL output the expr
results in higher precision than it computes them, thus causing lsb artifacts?
Is it possible to prevent that travesty, perhaps by setting something like a global precision variable or global output resolution variable or something like that?