1

Note: I've used the Matlab tag just in case they maintain the same precision. (From what I can tell both programs are very similar.)

As a follow-up to a previous question of mine (here), I'm trying to determine the level of precision I need to set (in a C++ program which I'm currently converting from Scilab code) in order mock the accuracy of the Scilab program. Essentially so both programs with produce the same (or very similar) results.

When computing a floating point calculation in Scilab, what is the level of precision maintained?

I've read (here and a few other places) that when working with floating point arithmetic in C++ a double can only maintain somewhere around 16 decimal places accurately, for example:

     4   8  12  16
     v   v   v   v
0.947368421052631578 long double
0.947368421052631526 double

How similar is this accuracy when compared to Scilab?

Community
  • 1
  • 1
Paul Warnick
  • 903
  • 2
  • 13
  • 26
  • It's not really valid to assess floating point error in "decimal places". What about 1.999999999999999999999999999 and 2.0? Those differ at all decimal places. Error depends on the *whole* number not just the decimal portion. – Suever Jun 07 '16 at 18:37
  • @Suever That's a good point. How do you suggest assessing them? – Paul Warnick Jun 07 '16 at 18:39
  • It really depends what you're trying to do. Are you trying to determine the correct datatype to use? – Suever Jun 07 '16 at 18:43
  • @Suever I've edited my question so that it might better explain my issue. – Paul Warnick Jun 07 '16 at 18:55
  • 1
    IEEE 754 double-precision floating point numbers are the standard representation in most common languages, like MATLAB, C++ and SciLab: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwiajY7LzZbNAhWJix4KHcrEA1wQFgggMAA&url=http%3A%2F%2Fforge.scilab.org%2Findex.php%2Fp%2Fdocscifloat%2Fdownloads%2Fget%2Ffloatingpoint_v0.2.pdf&usg=AFQjCNFQiOVdgkjuxhFXhp1PwDFY-J-Qbg&sig2=vH0cpadZqi0bNqa9F0Gmig&cad=rja so I don't expect you would need to do anything special to represent the precision, other than using C++ doubles (unless your SciLab code is using high-precision floats). – gariepy Jun 07 '16 at 19:05
  • Natively, yes. The document I linked to is an overview of floating point numbers in SciLab, which explicitly states they use the IEEE 754 standard. – gariepy Jun 07 '16 at 19:09
  • @gariepy I'm not exactly sure how to check what Scilab is using (the code just has variables declared as (for example): var1 = 1.3). Also, I am using doubles in C++ but unfortunately I'm receiving different results between Scilab and C++. Naturally I just assumed this was because Scilab was computing with a higher precision. – Paul Warnick Jun 07 '16 at 19:12
  • @gariepy My reasoning behind this was because my calculations (between C++ and Scilab) only begin to differ in very large loops. Whereas if both maintained the same precision, wouldn't the calculations remain the same? – Paul Warnick Jun 07 '16 at 19:14
  • 1
    No, they do not have to be the same. The floating point standard only guarantees about 16 significant digits of accuracy...beyond that, the extra digits are literally meaningless. – gariepy Jun 07 '16 at 19:23
  • 1
    For what it's worth, Scilab is open source, and you can browse the current sources [here](http://gitweb.scilab.org/?p=scilab.git;a=tree;f=scilab;h=8e5ab3c49ef1986f7ad183ae8bc1efba9a06ed90;hb=HEAD). From a brief look, I see a lot of C and C++ code using `double` for computations. So there's a strong likelihood that SciPy's floating-point type exactly matches the C double on the relevant platform. – Mark Dickinson Jun 16 '16 at 17:00
  • @MarkDickinson Yeah that's what I'm hoping. I'm trying to read the document below to confirm it. The problem is I have a loop in both the C++ and Scilab versions of the code that produce different results (using C++ doubles). I tried changing all my doubles to "long" and surprisingly it decrease the variance of the two results. That spark me asking the question of: Am I using the same data type here? (i.e. double precision in both languages). – Paul Warnick Jun 16 '16 at 17:08
  • From the document below: "In this section, we present the floating point numbers which are used in Scilab, that is, IEEE 754 doubles." (pg 28). I'm going to run through my code one more time to make sure I didn't type something incorrectly. If I can't find any errors I'm going to post another question asking something similar to "Why am I receiving different results?". Thanks again. – Paul Warnick Jun 16 '16 at 17:23
  • @gariepy In regards to your last comment. Are you saying that the answers will be the same except for after about 16 digits (once precision is lost)? Or do you mean that the answers can be completely different (i.e. one language output 1.1 and the other outputs 3.4)? – Paul Warnick Jun 16 '16 at 17:27
  • 1
    @PaulWarnick: answers will be the same up to 16 digits (because IEEE standard mandates that level of precision for double-precision floating point numbers) – gariepy Jun 16 '16 at 17:42

2 Answers2

2

Re-posting comment as an answer:

IEEE 754 double-precision floating point numbers are the standard representation in most common languages, like MATLAB, C++ and SciLab:

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwiajY7LzZbNAhWJix4KHcrEA1wQFgggMAA&url=http%3A%2F%2Fforge.scilab.org%2Findex.php%2Fp%2Fdocscifloat%2Fdownloads%2Fget%2Ffloatingpoint_v0.2.pdf&usg=AFQjCNFQiOVdgkjuxhFXhp1PwDFY-J-Qbg&sig2=vH0cpadZqi0bNqa9F0Gmig&cad=rja

so I don't expect you would need to do anything special to represent the precision, other than using C++ doubles (unless your SciLab code is using high-precision floats).

Note that the representations of two different IEEE 754 compliant implementations can differ after 16 significant digits:

MATLAB:

>> fprintf('%1.30f\n',1/2342317.0)
0.000000426927695952341190000000

Python:

>> "%1.30f" % (1/2342317,)
'0.000000426927695952341193713560'
gariepy
  • 3,576
  • 6
  • 21
  • 34
  • The final note in your answer is what gets me. Is this language specific? Meaning, will the numbers after 16 digits be the same for a specific calculation everytime you run the calculation if you're using the same language? Also, if after 16 digits the numbers can be different. How should I go about converting a program from one language (Scilab) to another (C++) while maintaining the same result? Is there some way to truncate the numbers so that only the first 16 digits are used (thus the calculations are the same)? – Paul Warnick Jun 07 '16 at 19:28
  • 1
    Not language-specific exactly, but platform/language specific, yes. So if I run the above statement in MATLAB 2015b on Windows, I get the same "extra digits" every time. However, I'm not guaranteed those same "extra digits" if I switch to MATLAB on Linux, or even a different version of MATLAB. – gariepy Jun 07 '16 at 19:30
  • 1
    But the point is, why are you worried about the "extra digits"? They aren't real. If you have an application that actually uses more than 16 significant digits ( and for real-world problems, these are very rare), then you need to be using one of the high-precision math libraries that most common languages have as extensions. But just trying to match arbitrary digits from one implementation to another is, well, a pointless endeavor. – gariepy Jun 07 '16 at 19:32
  • Thank you, this makes a lot more sense now. I guess my thinking was that Scilab was maintaining more precision than a C++ double. Whereas really it's the same (16 digits). – Paul Warnick Jun 07 '16 at 19:37
1

Every common language (standard C++, scilab, matlab, ...) use the same format for representing decimal numbers. It's known at IEEE754 and its precise documentation is explained in:

https://en.wikipedia.org/wiki/Double-precision_floating-point_format

It implies that the precision remains constant on nearly all common used system. It's a number close to 2^-52 (or equivalently 2.2204e-16). It defines the "distance from 1.0 to the next largest double-precision number".

When you're using scilab, you can confirm it with the %eps command https://help.scilab.org/docs/5.5.1/fr_FR/percenteps.html. For matlab, it's stored in the eps variable http://nl.mathworks.com/help/matlab/ref/eps.html. For C++, it's a bit harder (see: http://en.cppreference.com/w/cpp/types/numeric_limits/epsilon).

So, don't worry about precision if you do not use a specific machine (atypic architectures or very very old computers or high precision decimals (64bits double)). The default ones will always follow the same standard (IEEE 754).

But, don't forgot, even if it seems to be a constant, that error can is not the same between very high numbers and very small (the system is designed to have the best precision for the interval [0, 1[ and for the interval [1, MAXIMUM[).

It can be shown in the following example: 

>>> 1e100 == 1e100+1
True
>>> 1 == 2
False

To ensure that your code is portable for different languages, I suggest you to explicitly refer to the functions that give the machine precision. For instance, in scipy: print(np.finfo(float).eps). But, generally, well designed algorithms won't become a lot different on machine with slightly different epsilons.

For instance, if I implement a loop for something that will tends to be 0 (asymptoticly), in matlab, I should write: 

while(val < eps) do
...
end

So, the main advice should be: don't build an algorithm that would try to use too many information from the machine. Either you can use the real value of epsilon, either you can hard-code something like 2e-15. It would work on a lot of different machines.

Alexis Clarembeau
  • 2,776
  • 14
  • 27
  • 1
    This is a great answer but unfortunately the accepted answer clarified everything I was wondering about. – Paul Warnick Jun 07 '16 at 19:42
  • 2
    No problem :) I'll keep it in order to give extra information for further readings ^^ – Alexis Clarembeau Jun 07 '16 at 19:44
  • "use the same format for representing decimal numbers": actually they're binary, not decimal. – Simon Byrne Jun 08 '16 at 07:29
  • I am not a native English speaker but I wanted to use decimal to define a number with a decimal part (i.e after the dot) in order to contrast with integer numbers. Maybe I should use "double numbers" but it's more from "computer science" than from "common language". Am I wrong? – Alexis Clarembeau Jun 08 '16 at 07:48