0

I am reimplementing some Java code in C++ using JNI for speed improvements. To test my new code, I am comparing the outputs of the Java code with the C code. I have noticed that the C and Java codes are producing slightly different solutions when performing double calculations.

The problem comes when applying the sigmoid function, f(x) = 1 / (1 + e ^ -x), to a matrix. If I input a matrix containing 0.9595328833775463, Java produces 0.7230282708475686 and C produces 0.7230282708475684. I originally assumed it was just rounding when printing, but the binary doesn't match either.

I'm using Java 16.0.1 and Apple clang 12.0.5.

Specifically, this is the offending code:

Note: a and output are both 2d double arrays of dimensions height x width.

Java Version:

for (int i = 0; i < height; i++) {
    for (int j = 0; j < width; j++) {
        output[i][j] = (1.0 / (1 + Math.exp(-a[i][j])));
    }
}

C++ version:

for (int i = 0; i < height; i++) {
    for (int j = 0; j < width; j++) {
        output[i][j] = (1.0 / (1 + exp(-1 * a[i][j])));
    }
}

Edited question:

I'm aware there should be some error in the calculation, but I would like both of these calculations to have the same error. I have already validated my Java implementation and don't want to need to revalidate my code with the new errors. This is being used in neural net training and I'm not sure how the compounding errors will affect the outcome after a few million calculations. Is there a way I can calculate this in C in such a way that the outcomes match?

Botje
  • 26,269
  • 3
  • 31
  • 41
Brett
  • 59
  • 6
  • 8
    There's no reason to think that Java and C++ will use the same implementation of `exp`. Unlike basic arithmetic there's no expectation that such functions will produce the most accurate result possible, so different implementations will produce different results. – john Dec 01 '22 at 20:17
  • 3
    You're fighting over the 16th digit. Doubles are mud down there at the best of times. – user4581301 Dec 01 '22 at 20:20
  • `double` can support 15 to 17 significant digits. If we use the low side then your values are a match – NathanOliver Dec 01 '22 at 20:22
  • Will you accept that 16 digits is right at the limits of precision for `double` (15 to 17) so all you can expect is 15 digits which is what you have. – Richard Critten Dec 01 '22 at 20:49
  • See also e.g. https://stackoverflow.com/a/612640/869736; compilers (especially in C as opposed to Java) may use different algorithms and do their math at different levels of precision depending on how they're compiled. Some fudging at the very low digits is certainly reasonable and expected. – Louis Wasserman Dec 01 '22 at 20:54
  • 1
    With the single exception of `x = 0`, the mathematical expression `e^x`, where `x` is an exact `double` value will not give an exact `double` answer. The Java expression `Math.exp(x)` is defined to be within 1ULP of the exact mathematical value. Therefore, since the exact mathematical result will always be between two `double` values, the allowed calculation may return EITHER the next higher `double` or the next lower `double` - it does not have to return the closer one. This means that the return value of `Math.exp` is not precisely defined by its specification. – Dawood ibn Kareem Dec 01 '22 at 20:57
  • 3
    Reopened. The question isn't about floating-point math itself, it's about the difference in the requirements between C++ and Java. Java has some peculiar ideas about how floating-point math should be done, and it's not at all unreasonable to compare the two languages when interoperability is at issue. – Pete Becker Dec 01 '22 at 22:12
  • 2
    I think the only thing you can do is to look at the source code for `Math.exp` in the JDK version that you've got, and manually translate it to C++. I just looked at the source for `Math.exp` in a Coretto JDK, and as far as I can tell, it appears to approximate `Math.exp` with a series of quartic splines. The coefficients are all "magic numbers" and I have no idea how they would have been calculated. I haven't looked at the Oracle JDK version, but there's a good chance it uses a different approximation. – Dawood ibn Kareem Dec 01 '22 at 22:26
  • 1
    And you cannot hope for anything better in C++ when looking at the different library implementations. Translate the Java implementation you're using to C++ so that you know exactly what you are getting and pray there are no other subtle differences occurring outside of the code you can control. – user4581301 Dec 01 '22 at 22:29
  • 1
    I think you have a hard, and possibly unwinnable, battle if you want to get exactly the same floating point results from C or C++ and Java. Have a look at https://en.wikipedia.org/wiki/Comparison_of_Java_and_C%2B%2B , particularly the sixth bullet below the heading "semantics". [Note - check the cited references for accuracy; I haven't]. – Peter Dec 02 '22 at 01:58
  • I don't think that sixth bullet point is true of most Java versions. Typically, the "fast" implementation and the "strict" implementation are the same for most methods. – Dawood ibn Kareem Dec 02 '22 at 02:22
  • 1
    @DawoodibnKareem In 2016 OpenJDK [ported exp from fdlibm 5.3 to native Java](https://github.com/openjdk/jdk/commit/be91309965a532db4cf4fa794987753cfacbadd7#diff-86723ae3581601b28045840cded083f0cb64cd5d920149192e01d85f4f194e5f) It might be worth seeing of that C library produces the same output as its Java port. – Botje Dec 02 '22 at 08:53

0 Answers0