Simple code - error driving me crazy... float to 2 bytes and back

Question

Don't know how I finished school not being able to solve that little problem...

Following code produces unexpected output:

 public Color getPixel(int x, int y){
     return img.getPixelReader().getColor(x, y);
 }
 public float colorToHeight(Color c){
     int b =(int)(c.getBlue()*256);
     int r =(int)(c.getRed()*256*256);
     float h = (b+r)/2;
     h-=10000;
     return h;
 }
 public Color heightToColor(float h){
     h+=10000;
     h*=2;         
     double r= (int)(h/(256));
     double b = h-(r*256);
     return new Color(r/256,0,b/256,1f);
 }
public void debugPixel(int x, int z){

    System.out.println("test 424->Color:  "+heightToColor(424f));
    System.out.println("test Color->424:  "+colorToHeight(heightToColor(424f)));
    System.out.println("test Color->424->Color:  "+heightToColor(colorToHeight(heightToColor(424f))));
    System.out.println("check A: "+getPixel(x,z) + " Height: "+colorToHeight(getPixel(x,z)));
    System.out.println("check B: "+heightToColor(colorToHeight(getPixel(x,z))));
    System.out.println("check C: "+getPixel(x,z));
}

Output:

test 424->Color:  0x510070ff
test Color->424:  424.0
test Color->424->Color:  0x510070ff
check A: 0x5200cbff Height: 638.0
check B: 0x53001cff
check C: 0x5200cbff

I absolutely do not find the reason why [check B] != [check A]....Thanks!

Mind the `float <-> int` conversions, including the integral division `/`. It's quite **certain** the values won't be the same. — charlie, Aug 22 '16 at 14:01
Which `Color` class is this? (I assume that this class does some clamping internally, and particularly causes the "red" component, which is likely out of range, to be clamped to [0.0, 1.0]) — Marco13, Aug 22 '16 at 14:07
...I think I found it....thought it was a stupid error, the correct int-to-float color is (int)Math.round((c.getBlue()*255)), found it inside Color class. Now the error is only 1 in my checks, so there will be some rounding error somewhere. — SalkinD, Aug 22 '16 at 14:12
@SalkinD: No, your `int-to-float` was correct, or at least it didn't matter. The main error is realy the `float h = (b+r)/2;` line. Note that `(float) (1 + 2) / 2 == 1.0` and `1.0 * 2 == 2.0`, not the original `3.0`. The correct form is `float h = (b+r) / 2f`. — charlie, Aug 22 '16 at 14:35
thanks @charlie, true that. Both things (255 instead of 256) and this int/int division, now it seems to work! Thanks for all your help! — SalkinD, Aug 22 '16 at 16:39

Jean-François Fabre · Answer 1 · 2016-08-22T14:19:32.227

1

It can't be sure at all that you get the same values, because the back & forth operations (colorToHeight or heightToColor) involve floating point operations, hence loss of precision which happens there.

You can rewrite your methods without using floating point, not even divisions since all can be done with shifts.

 public int colorToHeight(Color c){
     int b = c.getBlue()>>8;
     int r = c.getRed()>>16;
     int h = (b+r)/2;  // will be an integer, b and r are even
     h-=10000;
     return h;
 }
 public Color heightToColor(int h){
     h+=10000;
     h*=2;         
     int r= h>>8;
     int b = h-(r<<8);
     return new Color(r>>8,0,b>>8,1f);
 }

edited Aug 22 '16 at 14:19

answered Aug 22 '16 at 13:27

Jean-François Fabre

137,073
23
153
219

might i ask where you do see a comparison in his code? – SomeJavaGuy Aug 22 '16 at 13:29
thats no pointer, thats the toString method of Color object its 0xRRGGBBFF – SalkinD Aug 22 '16 at 13:59
2

still he´s not comparing any references in the given code. My guess he rather meant why the output after `check A` and `check B` differ. – SomeJavaGuy Aug 22 '16 at 14:01
ok, you're right. Looked like pointers. My last edit (which was: even if you fix the pointers, you'll have the next error) explains the rounding/floating point error hence the different values. – Jean-François Fabre Aug 22 '16 at 14:13
Answer updated and improved. I think the DV is no longer justified (it was justified before!) – Jean-François Fabre Aug 22 '16 at 14:20

charlie · Accepted Answer · 2016-08-22T15:18:09.580

First, fix the division to:

float h = (b + r) / 2f;

The original code performs the integer division first, then it converts to float, e.g. (1 + 2) / 2 == 1.0. After the fix, it evaluates as (1 + 2) / 2.0 == 1.5.

Thus, in your second test case the height should be:

red = 83
blue = 29
height = 638.5

While with the incorrect integer division:

height = 638.0 // 0.5 truncated off
red = 83
blue = 28 // while 29 is expected

Regarding the RGB bounds, if the Color class operates on the range 0..255, take care when fixing that, since you need 2 distinct constants — an RGB upper LIMIT, and the composition SHIFT:

private static final int LIMIT = 255;
private static final int SHIFT = 256; // or whatever > LIMIT, e.g. 1000

public float colorToHeight(Color c){
    int b = (int) Math.round(c.getBlue() * LIMIT);
    int r = (int) Math.round(c.getRed() * LIMIT);
    float h = (b + r * SHIFT) / 2f;
    h -= 10000;
    return h;
}

public Color heightToColor(float h){
    h += 10000;
    h *= 2;         
    double r = (int) (h / SHIFT);
    double b = h - (r * SHIFT);
    return new Color(r / LIMIT, 0, b / LIMIT, 1f);
}

Thanks for your help and the solution summary! – SalkinD Aug 22 '16 at 16:41 — SalkinD, Aug 22 '16 at 16:41

Simple code - error driving me crazy... float to 2 bytes and back

2 Answers2