QVector2D and QVector3D value limitation?

Question

When I tried to use QVector2D or QVector3D, I got strange value, On my Mac the test code is below. I got stranger value.

QVector3D p2(858727.182, 40591545264.7, 0);

qDebug() <<  QString::number(p2.x(),'g', 12);

Output:

"858727.1875"
"40591544320"

why?

score 1 · Accepted Answer · answered Jun 23 '23 at 13:41

This answer will address the fact that 858727.182 "became" 858727.1875, and while not sure the following explanation is accurate, it will at least give you an idea about "why?".

First, QVector3D::x() returns a float.

Second, here are 2 tests using std::cout and std::setprecision:

std::cout<< std::setprecision(12) <<(float)858727.182<<std::endl;
std::cout<< std::setprecision(12) <<(double)858727.182<<std::endl;

And their outputs:

858727.1875
858727.182

Third, here's an IEEE 754 Floating Point conversion using IEEE-754 Floating Point Converter:

Value: 858727.182
Value actually stored in float : 858727.1875
Binary : 0 10010010 10100011010011001110011
Error due to conversion: 0.0055

What you're observing is the result of IEEE 754 standard of a binary32.

Same goes for QVector3D::y(), both numbers are too large for a float to store their exact values.

For more:

QVector2D and QVector3D value limitation?

1 Answers1