Anything related to double-precision floating-point arithmetic and data-types. Often used with reference to IEEE754 double-precision floating-point representation.
Questions tagged [double-precision]
240 questions
286
votes
28 answers
How do you round a double in Dart to a given degree of precision AFTER the decimal point?
Given a double, I want to round it to a given number of points of precision after the decimal point, similar to PHP's round() function.
The closest thing I can find in the Dart docs is double.toStringAsPrecision(), but this is not quite what I need…
Lucas Meadows
- 2,939
- 2
- 14
- 4
105
votes
11 answers
Should I use double or float?
What are the advantages and disadvantages of using one instead of the other in C++?
Khaled Alshaya
- 94,250
- 39
- 176
- 234
41
votes
8 answers
strange output in comparison of float with float literal
float f = 0.7;
if( f == 0.7 )
printf("equal");
else
printf("not equal");
Why is the output not equal ?
Why does this happen?
Ashish
- 8,441
- 12
- 55
- 92
40
votes
8 answers
How to correctly and standardly compare floats?
Every time I start a new project and when I need to compare some float or double variables I write the code like this one:
if (fabs(prev.min[i] - cur->min[i]) < 0.000001 &&
fabs(prev.max[i] - cur->max[i]) < 0.000001) {
continue;
}
Then…
Dmitriy
- 5,357
- 8
- 45
- 57
35
votes
3 answers
Does JavaScript have double floating point number precision?
I know it's an odd question, but does JavaScript have the capacity to work with double's as opposed to single floats? (64 bit floats vs. 32 bits.)
Travis
- 7,391
- 12
- 43
- 52
24
votes
5 answers
When using doubles, why isn't (x / (y * z)) the same as (x / y / z)?
This is partly academic, as for my purposes I only need it rounded to two decimal places; but I am keen to know what is going on to produce two slightly different results.
This is the test that I wrote to narrow it to the simplest…
Ben Pearson
- 7,532
- 4
- 30
- 50
22
votes
4 answers
Why does adding double.epsilon to a value result in the same value, perfectly equal?
I have a unit test, testing boundaries:
[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void CreateExtent_InvalidTop_ShouldThrowArgumentOutOfRangeException()
{
var invalidTop = 90.0 + Double.Epsilon;
new…
sprocket12
- 5,368
- 18
- 64
- 133
21
votes
4 answers
C++: doubles, precision, virtual machines and GCC
I have the following piece of code:
#include
int main()
{
if ((1.0 + 0.1) != (1.0 + 0.1))
printf("not equal\n");
else
printf("equal\n");
return 0;
}
When compiled with O3 using gcc (4.4,4.5 and 4.6) and run natively…
Jared Krumsie
- 397
- 1
- 8
- 17
19
votes
4 answers
Simulate tearing a double in C#
I'm running on a 32-bit machine and I'm able to confirm that long values can tear using the following code snippet which hits very quickly.
static void TestTearingLong()
{
System.Threading.Thread A = new…
Michael Covelli
- 2,113
- 4
- 27
- 41
16
votes
5 answers
Whats wrong with this simple 'double' calculation?
Whats wrong with this simple 'double' calculation in java?
I know some decimal numbers can not be represented in float / double binary formats properly, but with the variable d3, java is able to store and display 2.64 with no problems.
double d1 =…
vi.su.
- 685
- 2
- 9
- 19
15
votes
5 answers
Java BigDecimal precision problems
I know the following behavior is an old problem, but still I don't understand.
System.out.println(0.1 + 0.1 + 0.1);
Or even though I use BigDecimal
System.out.println(new BigDecimal(0.1).doubleValue()
+ new BigDecimal(0.1).doubleValue()
…
Jefferson
- 180
- 1
- 2
- 8
12
votes
3 answers
Comparing double values for equality in Java.
I would like some advice from people who have more experience working with primitive double equality in Java. Using d1 == d2 for two doubles d1 and d2 is not sufficient due to possible rounding errors.
My questions are:
Is Java's…
chr0mzie
- 181
- 1
- 1
- 9
11
votes
3 answers
Why c++ program compiled for x64 platform is slower than compiled for x86?
I've wrote program, and compiled it for x64 and x86 platform in Visual Studio 2010 on Intel Core i5-2500. x64 version take about 19 seconds for execution and x86 take about 17 seconds. What can be the reason of such behavior?
#include…
KolKir
- 859
- 1
- 8
- 16
10
votes
3 answers
64-bit Float Literals PHP
I'm trying to convert a 64-bit float to a 64-bit integer (and back) in php. I need to preserve the bytes, so I'm using the pack and unpack functions. The functionality I'm looking for is basically Java's Double.doubleToLongBits() method.…
kryo
- 716
- 1
- 6
- 24
10
votes
3 answers
Why is this bearing calculation so inacurate?
Is it even that inaccurate? I re-implented the whole thing with Apfloat arbitrary precision and it made no difference which I should have known to start with!!
public static double bearing(LatLng latLng1, LatLng latLng2) {
double deltaLong =…
Greg
- 765
- 2
- 9
- 17