is it a bug scaling 0.0-1.0 float to byte by multiplying by 255?

Question

this is something that has always bugged me when I look at code around the web and in so much of the literature: why do we multiply by 255 and not 256?

sometimes you'll see something like this:

float input = some_function(); // returns 0.0 to 1.0
byte output = input * 255.0;

(i'm assuming that there's an implicit floor going on during the type conversion).

am i not correct in thinking that this is clearly wrong?

consider this:

• what range of input gives an output of 0 ? (0 -> 1/255], right?
• what range of input gives an output of 1 ? (1/255 -> 2/255], great!
• what range of input gives an output of 255 ? only 1.0 does. any significantly smaller value of input will return a lower output.

this means that input is not evently mapped onto the output range.

ok. so you might think: ok use a better rounding function:

byte output = round(input * 255.0);

where round() is the usual mathematical rounding to zero decimal places. but this is still wrong. ask the same questions:

• what range of input gives an output of 0 ? (0 -> 0.5/255]
• what range of input gives an output of 1 ? (0.5/255 -> 1.5/255], twice as much as for 0 !
• what range of input gives an output of 255 ? (254.5/255 -> 1.0), again half as much as for 1

so in this case the input range isn't evenly mapped either!

IMHO. the right way to do this mapping is this:

byte output = min(255, input * 256.0);

again:

• what range of input gives an output of 0 ? (0 -> 1/256]
• what range of input gives an output of 1 ? (1/256 -> 2/256]
• what range of input gives an output of 255 ? (255/256 -> 1.0)

all those ranges are the same size and constitute 1/256th of the input.

i guess my question is this: am i right in considering this a bug, and if so, why is this so prevalent in code?

edit: it looks like i need to clarify. i'm not talking about random numbers here or probability. and i'm not talking about colors or hardware at all. i'm talking about converting a float in the range [0,1] evenly to a byte [0,255] so each range in the input that corresponds to each value in the output is the same size.

Answer 1

You are right. Assuming that valueBetween0and1 can take values 0.0 and 1.0, the "correct" way to do it is something like

byteValue = (byte)(min(255, valueBetween0and1 * 256))

Having said that, one could also argue that the desired quality of the software can vary: does it really matter whether you get 16777216 or 16581375 colors in some throw-away plot? It is one of those "trivial" tasks which is very easy to get wrong by +1/-1. Is it worth it to spend 5 minutes trying to get the 255-th pixel intensity, or can you apply your precious attention elsewhere? It depends on the situation: (byte)(valueBetween0and1 * 255) is a pragmatic solution which is simple, cheap, close enough to the truth, and also immediately, obviously "harmless" in the sense that it definitely won't produce 256 as output. It's not a good solution if you are working on some image manipulation tool like Photoshop or if you are working on some rendering pipeline for a computer game. But it is perfectly acceptable in almost all other contexts. So, whether it is a "bug" or merely a minor improvement proposal depends on the context.

Here is a variant of your problem, which involves random number generators: Generate random numbers in specified range - various cases (int, float, inclusive, exclusive) Notice that e.g. Math.random() in Java or Random.NextDouble in C# return values greater or equal to 0, but strictly smaller than 1.0.

You want the case "Integer-B: [min, max)" (inclusive-exclusive) with min = 0 and max = 256.

If you follow the "recipe" Int-B exactly, you obtain the code:

0 + floor(random() * (256 - 0))

If you remove all the zeros, you are left with just

floor(random() * 256)

and you don't need to & with 0xFF, because you never get 256 (as long as your random number generator guarantees to never return 1).

Answer 2

I think your question is misled. It looks like you start assuming that there is some "fairness rule" that enforces the "right way" of translation. Unfortunately in practice this is not the case. If you want just generate a random color, then you may use whatever logic fits you. But if you do actual image processing, there is no rule that says the each integer value has to be mapped on the same interval on the float value. On the contrary what you really want is a mapping between two inclusive intervals [0;1] and [0;255]. And often you don't know how many real discretization steps there will be in the [0;1] range down the line when the color is actually shown. (On modern monitors there are probable all 256 different levels for each color but on other output devices there might be significantly less choices and the total number might be not a power of 2). And the real mapping rule is that if for two colors red component values are R1 and R2 then proportion of the actual colors' red component brightness should be as close to R1:R2 as possible. And this rule automatically implies multiply by 255 when you want to map onto [0;255] and thus this is what everybody does.

Note that what you suggest is most probably introducing a bug rather than fixing a bug. For example the proportion rules actually means that you can calculate a mix of two colors R1 and R2 with mixing coefficients k1 and k2 as

Rmix = (k1*R1 + k2*R2)/(k1+k2)

Now let's try to calculate 3:1 mix of 100% Red with 100% Black (i.e. 0% Red) two ways:

1. using [0-255] integers Rmix = (255*3+1*0)/(3+1) = 191.25 ≈ 191
2. using [0;1] floating range and then converting it to [0-255] Rmix_float = (1.0*3 + 1*0.0)/(3+1) = 0.75 so Rmix_converted_256 = 256*0.75 = 192.

It means your "multiply by 256" logic has actually introduced inconsistency of different results depending on which scale you use for image processing. Obviously if you used "multiply by 255" logic as everyone else does, you'd get a consistent answer Rmix_converted_255 = 255*0.75 = 191.25 ≈ 191.