Questions tagged [floating-point-comparison]
29 questions
47
votes
6 answers
Floating point equality
It is common knowledge that one has to be careful when comparing floating point values. Usually, instead of using ==, we use some epsilon or ULP based equality testing.
However, I wonder, are there any cases, when using == is perfectly fine?
Look at…
geza
- 28,403
- 6
- 61
- 135
46
votes
1 answer
In C++, is exactly one of <, == and > guaranteed to be true on floats?
In C++, do I have a guarantee that, for any given float a and float b, one and only one of a < b, a == b and a > b is true?
If this differs between compilers and platforms, I am interested in Visual C++ on x86.
MacMillan
- 593
- 4
- 7
42
votes
3 answers
Why does Release/Debug have a different result for std::min?
Here is the test program:
void testFunc()
{
double maxValue = DBL_MAX;
double slope = std::numeric_limits::quiet_NaN();
std::cout << "slope is " << slope << std::endl;
std::cout << "maxThreshold is " << maxValue <<…
jpo38
- 20,821
- 10
- 70
- 151
42
votes
3 answers
What's the difference between identical(x, y) and isTRUE(all.equal(x, y))?
Is there any difference between testing isTRUE(all.equal(x, y)) and identical(x, y)?
The help page says:
Don't use 'all.equal' directly in 'if'
expressions — either use 'isTRUE(all.equal(....))' or
'identical' if appropriate.
but that "if…
mariotomo
- 9,438
- 8
- 47
- 66
36
votes
4 answers
Comparing floats in a pandas column
I have the following dataframe:
actual_credit min_required_credit
0 0.3 0.4
1 0.5 0.2
2 0.4 0.4
3 0.2 0.3
I need to add a column indicating where actual_credit >=…
darkpool
- 13,822
- 16
- 54
- 89
8
votes
1 answer
Python- How to make an if statement between x and y?
I've recently been breaching out to Python, as C++ is fun and all, but python seems kinda cool. I want to make Python do something as long as the input is between a certain number range.
def main():
grade = float(input("“What’s your…
Nate Dukes
- 183
- 1
- 1
- 6
5
votes
1 answer
Why can't I compare reals in Standard ML?
Why doesn't 1.0 = 2.0 work? Isn't real an equality type?
It gives the error:
Error: operator and operand don't agree [equality type required]
operator domain: ''Z * ''Z
operand: real * real
in expression:
1.0 = 2.0
Why won't…
sshine
- 15,635
- 1
- 41
- 66
4
votes
2 answers
What is the canonical way to check for approximate zeros in Catch2?
What is the canonical way to compare to approximate zeros in Catch2?
I found this way given a tolerance of 1e-12, but it is not clear it is the best way:
TEST("a approx. equal to b", "[test]"){
REQUIRE( a - b == (0_a).margin(1e-12) );
}
I am…
alfC
- 14,261
- 4
- 67
- 118
4
votes
2 answers
Checking if a specific float value is in list/array in Python/numpy
Care needs to be taken when checking for equality between floating point numbers, and should usually be done with a tolerance in mind, using e.g. numpy.allcose.
Question 1: Is it safe to check for the occurrence of a specific floating point number…
fromGiants
- 474
- 1
- 7
- 16
4
votes
2 answers
Why comparing two attributes type `float` and `int` with the same values get `False` in Python?
Let's consider the code below
code:
#!/usr/bin/env python
class Foo():
def __init__(self, b):
self.a = 0.0
self.b = b
def count_a(self):
self.a += 0.1
foo = Foo(1)
for i in range(0, 15):
foo.count_a()
print…
fronthem
- 4,011
- 8
- 34
- 55
2
votes
4 answers
Java List.contains object with double with tolerance
Let's say I have this class:
public class Student
{
long studentId;
String name;
double gpa;
// Assume constructor here...
}
And I have a test something like:
List students = getStudents();
Student expectedStudent = new…
JoeMjr2
- 3,804
- 4
- 34
- 62
2
votes
2 answers
How to evaluate whether two number are close enough or not in Python?
I have two numbers -
3.125000 Mbytes and 2.954880 Mbytes.
I want to compare them and it should return True since they are almost 3Mbytes. How can I do so in Python3.
I tried doing math.isclose(3.125000,2.954880, abs_tol=0.1).
However, this…
Prawn Hongs
- 441
- 1
- 5
- 17
2
votes
2 answers
Floating point equality for caching expensive computations
There are already a lot of questions and answers about the dangers of expecting two floats produced by separate computations to be exactly equal, because floating point numbers are not real numbers. This question is not about correctness contingent…
Joseph Garvin
- 20,727
- 18
- 94
- 165
1
vote
0 answers
Redis: using ZRANGE to search for a single value?
If in a Redis database a sorted set is filled with decimal values, how can I search in it for an exact value given as a decimal value, under the condition that exactly this decimal value was also used when filling the sorted set?
Since Redis does…
Amaterasu
- 153
- 10
1
vote
1 answer
What causes the different NaN behavior when compiling `_mm_ucomilt_ss` intrinsic?
Can someone explain me why the following code fails for GCC 8.5 with NaNs?
bool isfinite_sse42(float num)
{
return _mm_ucomilt_ss(_mm_set_ss(std::abs(num)),
_mm_set_ss(std::numeric_limits::infinity())) ==…
jerryct
- 13
- 3