14

Question in title.

I have a section of code:

double ccss = c * c + s * s;
double sqrtCCSS = sqrt(ccss);
if (sqrtCCSS != 0)
{
    n = n1 / sqrtCCSS;
}

and am just wondering if this is safe:

double ccss = c * c + s * s;
if (ccss != 0)
{
    n = n1 / sqrt(ccss);
}

My gut tells me yes, but floating point error is still somewhat mysterious.

Update:

At least for Python so far this seems impossible:

[ins] In [4]: x = np.nextafter(0, 1)
         ...: print(x)
         ...: print(np.sqrt(x))
5e-324
2.2227587494850775e-162

If the smallest possible float in numpy (given numpy is mostly in C, this seems relevant), cannot result in zero, the only possible number that would result in zero would have to be something that introduces grave floating point error but is not the smallest floating point number.

Update: I changed the variables to satisfy comments

phuclv
  • 37,963
  • 15
  • 156
  • 475
mas
  • 1,155
  • 1
  • 11
  • 28
  • 1
    Relevant: https://stackoverflow.com/questions/44374371/if-you-multiply-the-smallest-positive-floating-point-value-by-a-non-zero-number – mas Oct 07 '22 at 14:20
  • What language are you using? That may impact the behavior. – Some programmer dude Oct 07 '22 at 14:21
  • This particular codebase is C++14 but I would have expected floating point math to be near-universal. I guess that was an assumption. – mas Oct 07 '22 at 14:23
  • 3
    "I would have expected floating point math to be near-universal. I guess that was an assumption" - very much an assumption - not all languages use ieee754 math and even those that do sometimes have "interesting" differences - like C++ and Python for example. – Jesper Juhl Oct 07 '22 at 14:30
  • 7
    I can't see how sqrt(x) can be zero if x is not zero, given that, for numbers < 1, the root will be *larger* than the number itself. – Adrian Mole Oct 07 '22 at 14:35
  • @AdrianMole that is the theory I've been working with. I should have added it. – mas Oct 07 '22 at 14:36
  • ... but the issue is **proving that**, I guess. ;-) – Adrian Mole Oct 07 '22 at 14:39
  • Also, from [cppreference](https://en.cppreference.com/w/cpp/numeric/math/sqrt): *std::sqrt is required by the IEEE standard to be exact. ...* – Adrian Mole Oct 07 '22 at 14:42
  • 1
    I think `sqrt(xxyy)` should be `>0` but it can be small. eg. if `xxyyy=1e-30` then you will multibly with a large number what can lead to inaccuracies. As a rule of thumb I suggest `if (xxyy > epsilon)` where `epsilon` is small (10^-16 is fine for most of the time). – Botond Horváth Oct 07 '22 at 14:42
  • @AdrianMole: Re “Also, from cppreference: std::sqrt is required by the IEEE standard to be exact”: That would be a neat trick, considering most square roots of representable numbers are not representable, so exact results are impossible. IEEE 754 requires a conforming square root operation to produce correctly rounded results, not exact results. Avoid using cppreference.com. – Eric Postpischil Oct 07 '22 at 15:34
  • Please avoid using “x” (or any term) for one thing in your title and for a different thing in the sample code. – Eric Postpischil Oct 07 '22 at 15:35
  • When compiling with `-ffast-math` or something equivalent, you can probably have subnormals flush to zero. – chtz Oct 07 '22 at 15:35
  • Looks like cppreference.com needs to be updated from "exact" to "exact-ish". – Eljay Oct 07 '22 at 15:45
  • @Eric Well, the linked page does go on to say what is meant by "exact" in that context: *After rounding to the return type (using default rounding mode), the result of std::sqrt is indistinguishable from the infinitely precise result. In other words, the error is less than 0.5 ulp.* – Adrian Mole Oct 07 '22 at 15:46
  • 1
    In contrast to some math functions which use table lookups and interpolation. – Eljay Oct 07 '22 at 15:49
  • @AdrianMole: Re “‘In other words, the error is less than 0.5 ulp.’”: Well, that is wrong too, as a description of what correctly rounded results are. It happens to be true for square root, but there are cases for the other operations where the required result is indistinguishable from the “infinitely precise” result rounded to according to the rules for round-to-nearest-ties-to-even but the error is not less than ½ ULP. Those are the cases where the “infinitely precise” result result is exactly ½ ULP from representable values. And these things matter when writing proofs… – Eric Postpischil Oct 07 '22 at 16:06
  • … And that assumes the default rounding mode of to-nearest-ties-to-even without allowing for other rounding modes. Avoid using cppreference.com. – Eric Postpischil Oct 07 '22 at 16:07
  • 3
    The most dangerous aspect to me would be platforms using extra precision like x87. It could be that xxyy is not zero with extended precision, and that's used for the comparison, but then it is stored in memory and truncated to 64 bits, where it becomes 0. Avoiding this may require some strict standard compatibility flags, and may not even be implemented by all compilers. – Marc Glisse Oct 07 '22 at 16:07
  • @Eljay: I would not call the common implementations of math library functions like sine or logarithm “table lookups and interpolation”. Some parts of some of them may use some table lookup, but the bulk is usually a partitioning of the domain in a more sophisticated way, if that is necessary. (E.g., tiny, small, medium, large, and very large values for sine, so not a regular table.) And the evaluation within a partition is typically done with a polynomial specifically engineered for its interval, not with a general interpolation between the endpoints. – Eric Postpischil Oct 07 '22 at 16:10
  • Restating my comment, with extended precision and without some strict mode, it is possible to have both x!=y and x==y return the same value... – Marc Glisse Oct 07 '22 at 16:14
  • @EricPostpischil why do you delete your answer? – phuclv Oct 07 '22 at 16:28
  • @phuclv: OP used `x` for two different things; they used `sqrt(x*x + y*y)` in the question but `sqrt(x)` in the title. The former can be zero when `x` is positive, and I wrote the answer based on that. But that is not what they meant to ask about. – Eric Postpischil Oct 07 '22 at 17:17
  • @EricPostpischil changed. – mas Oct 07 '22 at 17:45
  • mas, `sqrt(ccss)` can have the return value of zero and division by 0: "if the value of the second operand is zero, the behavior is undefined." The `if()` test is a good idea. – chux - Reinstate Monica Nov 02 '22 at 02:19

1 Answers1

10

and am just wondering if this is safe:

double xxyy = x * x + y * y;
if (xxyy != 0)
{
    n = n1 / sqrt(xxyy);
}

It's always safe because floating-point math doesn't trap and you can freely divide by zero or even Inf/NaN, unless you tell the compiler to trap on those cases

Anyway if you just want to check whether the denominator is zero then the question on the title is actually different from what you're talking about in the question's content. The answer to the question

Given x>0, is it possible for sqrt(x) to be zero?

is yes, if denormals-are-zero (DAZ) and/or flush-to-zero (FTZ) are turned on. In fact most non-x86 architectures have DAZ/FTZ enabled by default, or don't even support denormals and always turn DAZ/FTZ on, because operation on denormal numbers are very slow (see Why does changing 0.1f to 0 slow down performance by 10x?). In x86 you can also turn those flags on and some compilers will turn them on by default

For example the below sample code

f = 0x0Cp-1022;
_MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON);
_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
printf("%a %a\n", f, sqrt(f));

may print 0x0.000000000000cp-1022 0x0p+0 because f is treated as zero by sqrt due to DAZ

Some demo on Godbolt. And gcc demo on ideone

The above snippet can also print 0x0p+0 0x0p+0 because f is treated as zero by printf due to FTZ even though it contains a non-zero denormalized value

If you set DAZ/FTZ before assigning f like this

_MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON);
_MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON);
f = 0x0Cp-1022;
printf("%a %a\n", f, sqrt(f));

then it'll likely also print 0x0p+0 0x0p+0, but in this case f can be assigned an actual zero value instead, and printing its bit pattern will result in all zero bits


However if you've asked this

Given x and y which aren't both zero at the same time, is it possible for sqrt(x*x + y*y) to be zero

then the answer to that question is yes even if DAZ/FTZ aren't enabled, because the numbers can be so small that their product became underflow. Try something like this and see

double x = 0x1p-1021;
double y = 0x2p-1020; // or y = 0
double xxyy = x * x + y * y;
printf("%a %a\n", x, y);
printf("%a %a\n", xxyy, sqrt(xxyy));
phuclv
  • 37,963
  • 15
  • 156
  • 475
  • "yes, if DAZ and/or FTZ are turned on": maybe clarify that they have to change from off to on between the moment x is computed and the moment sqrt(x) is computed? Then it couldn't happen for the ccss variable in the OP's example, since the mode doesn't change in between, and if it flushed to zero, ccss would already be 0? Or can you tweak things to make even that fail? – Marc Glisse Oct 08 '22 at 08:01
  • @MarcGlisse that isn't important, you can turn them on at the start of the program or right before `sqrt`, in both cases `sqrt(x)` will be zero because the inputs are both zero https://godbolt.org/z/jn4vx57xx. Anyway I've edited the question and clarified – phuclv Oct 08 '22 at 09:31
  • My point is that using memcpy is "cheating", in the OP's case ccss gets its value from an addition, not memcpy, which may restrict the possibilities. If that's the case, it can be useful to know. – Marc Glisse Oct 08 '22 at 10:10
  • @MarcGlisse I couldn't use a simple assignment in such a short demo due to compiler optimization which ignores the DAZ/FTZ flag at runtime and just prints out the precomputed constant, that's why I copy the bit pattern directly. But in a larger program where the compiler doesn't precompute the result then it's what really happens – phuclv Oct 08 '22 at 10:49
  • if `c*c+s*s` underflows to zero in double precision, but that intermediate value is kept with extra precision that remain > 0, the assumption then relies on the fact that the assignment `double ccss=c*c+s*s` shall always force a conversion to double precision. Has this always been the case in C++ standard? – aka.nice Oct 09 '22 at 10:09
  • @aka.nice assignment and cast always strip the extra precision, so `double ccss=c*c+s*s` must cast to double. The type of temporary expressions however depends on `FLT_EVAL_METHOD`, if it's 0 then operations must always be done in `double` without any extra precision – phuclv Oct 09 '22 at 11:40
  • `$ gcc --version gcc (GCC) 12.2.0` invoked with no flags: `warning: division by zero [-Wdiv-by-zero] 31 | std::cout << 1/0; | ~^~ Floating point exception (core dumped` – Vorac Oct 22 '22 at 08:31
  • @Vorac what do you mean? How on earth is that related to the answer/question? Obviously `1/0` is integer division by 0 which always trap in all modern platforms. Only `1.0/0` or `1/0.0` is floating point division where zero is allowed – phuclv Oct 22 '22 at 15:16
  • @phuclv sorry for the silly oversight! Unfortunately I cannot upvote your answer a second time. – Vorac Oct 23 '22 at 06:39
  • 1
    "... freely divide by zero" is not supported in C as "if the value of the second operand is zero, the behavior is undefined." I suspect C++ inherit this. – chux - Reinstate Monica Nov 02 '22 at 02:22