10

I am trying to make equivalence tests on an algorithm written in C++ and in Matlab. The algorithm contains some kind of a loop in time and runs more than 1000 times. It has arithmetic operations and some math functions.

I feed the initial inputs to both platforms by hand (like a=1.767, b=6.65, ...) and when i check the hexadecimal representations of those inputs they are the same. So no problem for inputs. And get the outputs of c++ to matlab by a text file with 16 decimal digits. (i use "setprecision(32)" statement)

But here comes the problem; although after the 614'th step of both code, all the results are exactly the same, on the step of 615 I get a difference about 2.xxx..xxe-19? And after this step the error becomes larger and larger, and at the end of the runs it is about 5.xx..xxe-14.

0x3ff1 3e42 a211 6cca--->[C++ function]--->0x3ff4 7619 7005 5a42

0x3ff1 3e42 a211 6cca--->[MATLAB function]--->ans

ans - 0x3ff4 7619 7005 5a42

= 2.xxx..xxe-19

I searched how matlab behaves the numbers and found really interesting things like "denormalized mantissa". While realmin is about e-308, by denormalizing the mantissa matlab has the smallest real number about e-324. Further matlab holds many more digits for "pi" or "exp(1)" than that of c++.

On the other hand, matlab help says that whatever the format it displays, matlab uses the double precision internally.

So,I'd really appreciate if someone explains what the exact reason is for these differences? How can we make equivalence tests on matlab and c++?

crbah
  • 338
  • 4
  • 12
  • "It has arithmetic operations and some math functions." What kind of maths functions? – mathematician1975 Jun 22 '12 at 07:14
  • You might want to take note that the precision of C++ is a factor of the quality of your library implementation. There is a certain amount of unavoidable imprecision in the least significant bits. – DevSolar Jun 22 '12 at 07:18
  • What do you mean by text file with 16 bit precision? that is 2 characters. If you mean binary 16 bits it is still very small. I would be careful about going through a text file - save the full variable to a binary file and load it in matlab to be sure. – robince Jun 22 '12 at 07:42
  • @robince, sorry, my bad, by "16bit precision" i meant "double values written with 16 decimal digits". i will correct there. – crbah Jun 22 '12 at 07:58
  • @mathematician1975, math functions like "sin, cos, exp" and some real numbers like sqrt(2),sqrt(5) or pi – crbah Jun 22 '12 at 08:00
  • @user1474013 I think that is probably part of the problem. I don't think 16 decimal digits is enough - some round off error could be coming from that. I would suggest saving the binary values for a fair comparison. – robince Jun 22 '12 at 21:25

3 Answers3

11

There is one thing in x86 CPU about floating points numbers. Internally, the floating point unit uses registers that are 10 bytes, i.e. 80 bits. Furthermore, the CPU has a setting that tells whether the floating point calculations should be made with 32 bits (float), 64 bits (double) or 80 bits precision. Less precision meaning faster executed floating point operations. (The 32 bits mode used to be popular for video games, where speed takes over precision).

From this I remember I tracked a bug in a calculation library (dll) that given the same input did not gave the same result whether it was started from a test C++ executable, or from MatLab.. Furthermore, this did not happen in Debug mode, only in Release!

The final conclusion was that MatLab did set the CPU floating point precision to 80 bits, whereas our test executable did not (and leave the default 64 bits precision). Furthermore, this calculation mismatch did not happen Debug mode because all the variables were written to memory into 64 bits double variables, and reloaded from there afterward, nullifying the additional 16 bits. In Release mode, some variables were optimized out (not written to memory), and all calculations were done with floating point registers only, on 80 bits, keeping the additional 16 bits non-zero value.

Don't know if this helps, but maybe worth knowing.

Didier Trosset
  • 36,376
  • 13
  • 83
  • 122
  • That would have been an extremely fun bug to find. – Li-aung Yip Jun 22 '12 at 08:17
  • 1
    and is there a way to set matlab to use 64-bit precision? – crbah Jun 22 '12 at 08:21
  • *Less precision meaning faster executed floating point operations.* -- Not exactly true. The precision modes are for compatibility with IEEE standard, calculations (usually?) use the same number of cycles; however, doubles mean more data to be moved and less parallelization in SIMD registers. – peterchen Jun 22 '12 at 08:41
4

A similar discussion occurred before, the conclusion was that IEEE 754 tolerates error in the last bit for transcendental functions (cos, sin, exp, etc..). So you can't expect exactly same results between MATLAB and C (not even same C code compiled in different compilers).

Community
  • 1
  • 1
Amro
  • 123,847
  • 25
  • 243
  • 454
  • yeah that's great to see. it is really weird that such two "well known" platforms calculate functions in different ways leading different results. unfortunately after each time step in my algorithm, this last-bit difference comes closer to more significant bits, as in all "unstable" codes. – crbah Jun 23 '12 at 07:38
  • @user1474013: my guess is that those round-off errors are accumulating over iterations, making the difference bigger and bigger... If your application needs more precision than what is provided, consider using arbitrary-precision libraries (MATLAB has the Symbolic toolbox) – Amro Jun 23 '12 at 07:50
  • I am using Matlab+Simulink. I can't use Variable Precision Arithmetic (vpa) Toolbox because in Simulink there is no option for sin(), cos() functions to give results using vpa. – crbah Jun 23 '12 at 13:48
  • While debugging in both C++ and MATLAB, after 2053rd step i found out that my angle _alpha_ is calculated exactly the same in C++ and MATLAB, but the _sin()_ functions in MATLAB and C++ give two different results! [c++]=0x3......4b, [matlab]=0x3......4a – crbah Jun 23 '12 at 13:54
0

I may be way off track here and you may already have investigated this possibility but it could be possible that there are differences between C++ and Matlab in the way that the mathematical library functions (sin() cos() and exp() that you mention) are implemented internally. Ultimately, some kind of functional approximation must be being used to generate function values and if there is some difference between these methods then presumably it is possible that this manifests itself in the form of numerical rounding error over a large number of iterations.

This question basically covers what I am trying to suggest How does C compute sin() and other math functions?

Community
  • 1
  • 1
mathematician1975
  • 21,161
  • 6
  • 59
  • 101