I am using Eigen Library to do some matrix calculations.
I have such calculation
MatrixXf A = MatrixXf::Random(3, 3);
MatrixXf B = A.inverse();
MatrixXf C = A*B;
MatrixXf D = C - Matrix3f::Identity();
The result I got for C is:
1 0 1.19209e-07
1.19209e-07 1 0
-1.19209e-07 0 1
and for D is
0 0 1.19209e-07
1.19209e-07 0 0
-1.19209e-07 0 1.19209e-07
I would like to reduce this error caused by the floating-point precision. the reason is that I need to multiply a very big coefficient (e+09) onto this matrix which amplifies these errors significantly. I tried using double precision, but by multiplying with (e+09), the error become a real problem by twice the multiplication.