5

For a number x in the binary floating point number system, units in the last place is defined as ulp(x) = 2^{E(x)} * 2^{1 - p} = 2^{E(x)}*epsilon, where the exponent is the exponent of the base 2 used to represent x, p is the precision of the number format (53 for the double precision format), and epsilon = 2^{1-p} is the machine tolerance.

Looking at std::numeric_limits I expected to find something like std::numeric_limits<double>::ulp(x), but this seems to be unavailable. Or something like std::numeric_limits<double>::exponent(x) that I can multiply with std::numeric_limits<double>::epsilon() but it is not there. The ulp function seems to be available in boost.

I can code something myself, but I would much rather use standardized code, if there is one. Did I miss something in the documentation?

Edit: to make it more clear, I am talking about the calculation of the exponent of a floating point number x, not getting min_ or max_ exponent range for a floating point number format (type).

tmaric
  • 5,347
  • 4
  • 42
  • 75
  • Did You look at: https://en.cppreference.com/w/cpp/types/numeric_limits ? – Robert Andrzejuk Jan 31 '19 at 12:37
  • 3
    Yep, that's why the question is full with `std::numeric_limits` references. Where is the `ulp` or `exponent` in `numeric_limits`? Not `min_` or `max_` exponent, exponent of a floating point number `x`. – tmaric Jan 31 '19 at 12:38
  • @tmaric what do you mean by "exponent"? – eerorika Jan 31 '19 at 12:40
  • @eerorika: d0.d1d2d3d4...d_{p-1}*2^E, E is the exponent. I need the exponent, or the `ulp`, it makes no difference. – tmaric Jan 31 '19 at 12:43
  • 1
    That is not a correct definition of the “epsilon.” The “epsilon” is the distance from 1 to the next greater representable number, but the minimum *g* such that one plus *g* exceeds one is slightly over one half ULP, while round-to-nearest-ties-to-even is in effect. – Eric Postpischil Jan 31 '19 at 12:43
  • @EricPostpischil: if g is in the floating point system, then g+1.0 is in the floating point system, and if g = min number such that this is true, then g is the epsilon. It is 2^0 * 2^{-p+1}, or 1 * ulp(1.0). Rounding mode only determines the relative error between the real and floating point number, in terms of the epsilon. It is ~0.5 epsilon for round to nearest. – tmaric Jan 31 '19 at 12:46
  • C code to find the ULP is [here](https://stackoverflow.com/questions/53598657/is-there-a-correct-constant-expression-in-terms-of-a-float-for-its-msb). – Eric Postpischil Jan 31 '19 at 12:46
  • 1
    @tmaric: In IEEE-754 basic 64-bit binary, 1 is representable (and its ULP or LSB is 0x1p-52) and 0x1p-53+0x1p-105 is representable, but their sum using exact mathematics, 1+0x1p-53+0x1p-105, is not representable. Using floating-point arithmetic with round-to-nearest-ties-to-even, their sum is 1+0x1p-52. This is the *g* as defined in the question, since the next smaller representable value is 0x1p-53, and adding it to 1 yields 1 because the mathematical sum is midway between representable values, so it is rounded down to the even-low-bit 1. – Eric Postpischil Jan 31 '19 at 12:52
  • 2
    Re “I am talking about the calculation of the exponent of a floating point number x”: The `std::frexp` function may be used for getting the exponent. This is different from getting the ULP, due to effects with subnormal numbers. – Eric Postpischil Jan 31 '19 at 12:54
  • @EricPostpischil: this is how I understood this stuff: |round(x) - x| < ulp(x), the absolute rounding error. For *normalized numbers*, the relative error |round(x) - x| / |x| < ulp(x) / |x|. Further, |x|>2^E, where E is the exponent of the lower (left on the floating point number line) floating point number of x. Since ulp(x) = 2^E(x) * 2^{-p + 1} = 2^E(x) * epsilon, the upper bound of the relative error is `epsilon = 2^{-p+1}`. With rounding to nearest then I have a relative error 0.5 * epsilon. Ah, now I understand what you mean, and my definition of `g` is wrong. – tmaric Jan 31 '19 at 13:08
  • @EricPostpischil: for denormalized numbers, can I check if the number is outside of the normalized range and use `2^{Emin}*epsilon` for the `ulp`? – tmaric Jan 31 '19 at 13:15
  • 1
    @tmaric: Yes, if the absolute value of the number is less than misnamed `std::numeric_limits::min()` (which is the minimum normalized value, rather than the minimum value), then the ULP is that `std::numeric_limits::min()` multiplied by the epsilon. – Eric Postpischil Jan 31 '19 at 13:22
  • @EricPostpischil I would use `fpclassify` to check for normalisation. – eerorika Jan 31 '19 at 14:09
  • @eerorika: To what end? Whatever method `fpclassify` uses to test the number and return a value, it must then be compared with `FP_SUBNORMAL` and `FP_ZERO`. It would seem difficult for that to be better than taking the absolute value and performing one comparison. – Eric Postpischil Jan 31 '19 at 14:43
  • @EricPostpischil mostly because you probably need to test the other classes anyway if you want to handle them correctly. Infinite, NaN in particular. – eerorika Jan 31 '19 at 14:49
  • @eerorika: `fpclassify` must either perform its own floating-point comparisons and then branch to produce integers, which must then be tested again, or it must examine the bit fields in the floating-point representation and then branch to produce integers, which must then be tested again. I am skeptical this is better than taking the absolute value and performing direct comparisons against the minimum normal and infinity. – Eric Postpischil Jan 31 '19 at 14:54

3 Answers3

1

The standard library indeed does not have an ulp function.

2^{E(x)}*epsilon

Can be calculated with:

int exp;
std::frexp(std::fabs(x), &exp);
double result = std::ldexp(std::numeric_limits<double>::epsilon(), exp - 1);

Caveat: This is only correct for normal floating point values. For denormal values there might not even exist a floating point value that can represent ULP.


I expected to find something like std::numeric_limits<double>::ulp(x), but this seems to be unavailable. Or something like std::numeric_limits<double>::exponent(x)

numeric_limits contains only constants (or functions which return constants). It doesn't have any functions that perform calculations and thus no functions which take input values. <cmath> header has functions for calculations.


As a sidenote to readers in general: If your intention is to find the next/previous representable value, use nextafter family of functions instead of adding/subtracting ULP.

eerorika
  • 232,697
  • 12
  • 197
  • 326
  • 3
    `std::ldexp` should be used rather than `std::pow` for manipulations of the floating-point exponent, as it is designed for the purpose and avoids potential rounding errors inherent to the more complicated power function. – Eric Postpischil Jan 31 '19 at 13:11
  • @EricPostpischil added to answer. – eerorika Jan 31 '19 at 13:14
  • 3
    Attempting to calculate the ULP by multiplying the exponent obtained by `std::frexp` with the “epsilon” will fail for subnormal numbers, as their ULP is clamped at the ULP of the lowest normal value while `std::frexp` continues to report lower exponents for them. – Eric Postpischil Jan 31 '19 at 13:15
  • 2
    Using `std::pow(2, exp) * std::numeric_limits::epsilon();` for the ULP is off by one power of two because `std::frexp` reports an exponent scaled to put the significand in [½, 1), rather than the more common [1, 2). – Eric Postpischil Jan 31 '19 at 13:16
  • Also note that the exponent returned by `frexp` is off by 1, since it adjust the mantissa to a `0.1...` number. – Christian Rau Jan 31 '19 at 13:17
  • @ChristianRau fixed. – eerorika Jan 31 '19 at 13:26
  • @eerorika A link to https://en.cppreference.com/w/cpp/numeric/math/frexp could be added? – Robert Andrzejuk Jan 31 '19 at 15:39
1

I found an alternative, it doesn't fit into comments, so I'm posting an answer.

I think it can also be done by using std::nextafter and std::nexttoward:

template<typename FT> 
std::enable_if_t<std::is_floating_point_v<FT>, FT> 
ulp(FT x)
{
    if (x > 0)
        return std::nexttoward(x, std::numeric_limits<FT>::infinity()) - x;
    else 
        return x - std::nexttoward(x, -std::numeric_limits<FT>::infinity());
}

This is valid if nexttoward results in a floating point number that is next toward INF if the number is positive, and toward -INF if the number is negative. Because this number is a floating point number, the distance from x is going to be ulp(x) and it doesn't have a rounding error.

Here is a test program for this idea, it seems to work without using power functions and computing exponents manually (it's probably doing something like that under the hood). I tested it by checking:

// Check ulp by adding 0.5ulp to x and let the rounding to nearest round to x. 
assert((x + (0.5 *ulpx) == x));

Here is the MWE for it:

#include <cmath>
#include <cassert>
#include <iostream>
#include <iomanip>

using namespace std;

template<typename FT> 
std::enable_if_t<std::is_floating_point_v<FT>, FT> 
ulp(FT x)
{
    if (x > 0)
        return std::nexttoward(x, numeric_limits<FT>::infinity()) - x;
    else 
        return x - std::nexttoward(x, -numeric_limits<FT>::infinity());
}

void test_ulp(double x)
{
    double ulpx = ulp(x);

    std::cout << "ulpx = " << std::setprecision(25) << ulpx << std::endl;

    double xNext = x + ulpx; 

    // Check ulp by adding 0.5ulp to x and let the rounding to nearest round to x. 
    if (x > 0)
        assert((x + (0.5 *ulpx) == x));
    else 
        assert((x - (0.5 *ulpx) == x));

    std::cout << std::setprecision(25) <<  x << " + " << ulpx 
        << " == " << xNext << std::endl; 
}

int main()
{
    // Denormalized test. 
    test_ulp(0.); 

    // Epsilon test. 
    test_ulp(1.); 

    // Random number test. :) 
    test_ulp(42.); 

    // Epsilon test. 
    test_ulp(-1.); 
}

Edit: tried to find what nexttoward does, looked into gcc STL source code and only found this:

#ifndef __CORRECT_ISO_CPP11_MATH_H_PROTO_FP
   constexpr float
   nexttoward(float __x, long double __y)
   { return __builtin_nexttowardf(__x, __y); }

   constexpr long double
   nexttoward(long double __x, long double __y)
   { return __builtin_nexttowardl(__x, __y); }
 #endif

 #ifndef __CORRECT_ISO_CPP11_MATH_H_PROTO_INT
   template<typename _Tp>
     constexpr typename __gnu_cxx::__enable_if<__is_integer<_Tp>::__value, 
                                               double>::__type
     nexttoward(_Tp __x, long double __y)
     { return __builtin_nexttoward(__x, __y); }
 #endif
tmaric
  • 5,347
  • 4
  • 42
  • 75
  • 2
    For -1, the next number in the direction of infinity is -1+0x1p-53, but the ULP of -1 is 0x1p-52. When going to a lower-magnitude binade, the distance to the first number in the lower binade is half the size of the ULP of the starting number. – Eric Postpischil Jan 31 '19 at 19:44
  • @EricPostpischil: thanks! I have switched the direction for negative numbers towards -INF. – tmaric Feb 01 '19 at 08:18
0

Technically, solution could be to extract the exponent bits, and subtract 1074 from it. This gives the exponent of ulp. Then scale the sign by this exponent if you want a signed ulp. Something like

int exp = extract_exponent_bits(x):
Int sign = 1 - 2* extract_sign_bit(x);
return ldexp((double)sign,exp-1074);

Bit extraction left as an exercize, pointer aliasing like in sun fdlibm is not an option nowadays, so just memcpy to an uint64, shift >> 52, etc...

Generalization to non ieee might be more involved, but are you going to need it?

aka.nice
  • 9,100
  • 1
  • 28
  • 40