0

i have an XYSeries that holds Lat and Lon position of points. in some point i calculate the center point of all those positions. the code is very simple:

for(int i = 0; i < pnts.getItemCount(); i++){
                avgLon += (double)pnts.getX(i);
                avgLat += (double)pnts.getY(i);
            }
            cntrMass = new Vector2D(avgLon/pnts.getItemCount(), avgLat/pnts.getItemCount());

however this average calculation is not precise. when i calculate the same thing using excel sheet and the same data set there is a difference in the center point. I am using the exact number of significant digits for both calculations. the differences is of a 1e-6 magnitude, when converted to meters it gets pretty significant. any idea of how to fix this problem? any help would be appreciated.

  • 2
    have you considered that (maybe) it's your code that makes it less precise? what data types are you using? – Stultuske Apr 06 '16 at 08:16
  • what's the expected output and what does your code produces? – phuclv Apr 06 '16 at 08:19
  • 2
    see http://stackoverflow.com/questions/322749/retain-precision-with-double-in-java - For better prcision you can use BigDecimal – Pooya Apr 06 '16 at 08:19
  • @Stultuske in the XYSeries (the pnts variable) the type is number, everything else is double. –  Apr 06 '16 at 08:20
  • This is due to the limited precision of the binary representation a computer uses for everything. For example, you cannot represent 0.1 in binary with limited memory. See also http://stackoverflow.com/questions/1089018/why-cant-decimal-numbers-be-represented-exactly-in-binary – Stanley F. Apr 06 '16 at 08:21
  • @LưuVĩnhPhúc from Excel `32.7873493332460 35.0055739161479` and from java `32.7873495192755 35.0055742717924` –  Apr 06 '16 at 08:21
  • @StanleyF. there's no value exact to 0.1 in double but that's not the problem here. double can be precise to ~15 digits and Excel also uses double internally – phuclv Apr 06 '16 at 08:30
  • Would you mind sharing the concrete input with us that led to the stated result? – Alexander Apr 06 '16 at 08:33
  • @Alexander its too long for this comment –  Apr 06 '16 at 08:34
  • @Danny Lavrov you could edit your Question. – Alexander Apr 06 '16 at 08:35
  • i am trying to put the number in two columns but the editor wont let me. –  Apr 06 '16 at 08:38
  • I don't think it's the duplicate of the other question, because Excel also uses double internally. It has no bigdecimal support. The OP needs to create a [Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve) so that the question can be reopened – phuclv Apr 07 '16 at 02:12
  • @DanyLavrov you don't put the whole code here. You create an [SSCCE](http://sscce.org/) – phuclv Apr 07 '16 at 02:13

2 Answers2

0

You are using doubles. If you cast your super precise number to a double you lose some precision. Then if you do it again with another number and then do some calculation using those already rounded numbers you will get a result that is rounded again and therefore even more imprecise.

You can get more precise results using a datatype like BigDecimal, but you will never avoid rounding errors.

And the Vector2D uses doubles to store the values too so you get the same issue there.

Jim
  • 995
  • 2
  • 11
  • 28
0

Here is example of how to use BigDecimal given your code:

int itemCount = pnts.getItemCount();
int scale = 3;
BigDecimal itemCountBig = new BigDecimal(itemCount);
BigDecimal avgLonBig = BigDecimal.ZERO.setScale(scale, BigDecimal.ROUND_HALF_UP);
BigDecimal avgLatBig = BigDecimal.ZERO.setScale(scale, BigDecimal.ROUND_HALF_UP);
for(int i = 0; i < itemCount; i++){
    avgLonBig = avgLonBig.add( new BigDecimal(String.valueOf((double)pnts.getX(i))));
    avgLatBig = avgLatBig.add( new BigDecimal(String.valueOf((double)pnts.getY(i))));
}
cntrMass = new Vector2D(
    avgLonBig.divide(itemCountBig, MathContext.DECIMAL128).setScale(scale, BigDecimal.ROUND_HALF_UP).doubleValue(),
        avgLatBig.divide(itemCountBig, MathContext.DECIMAL128).setScale(scale, BigDecimal.ROUND_HALF_UP).doubleValue());

Perhaps you can give it a try and see if it helps with your precision issue.

craigsparks
  • 134
  • 6