0

I would ideally like to store each component of normals and tangents in 10 bits each, and a format supported by graphics APIs is A2R10G10B10 format. However I don't understand how it works. I've seen questions such as this which show how the bits are laid out. I understand how the bits are laid out, but what I don't get is how the value is fetched and interpreted when unpacked by the graphics API (Vulkan/OpenGL).

I want to store each component in 10 bits, and read it from the shader as signed normalised (-1.f to 1.f), so I'm looking at VK_FORMAT_A2B10G10R10_SNORM_PACK32 in Vulkan. Is one of the bits of the 10 bits used to store the sign for the value? How does it know if it's a negative or positive value? For a 8, 16, 32 etc bit number the first bit represents its signedness. How does this work for a 10-bit number? Do I have to manually use two's complement to form the negative version of the value using the ten bits?

Sorry if this is a dumb question, I can't understand how this works.

Zebrafish
  • 11,682
  • 3
  • 43
  • 119
  • "*For a 8, 16, 32 etc bit number the first bit represents its signedness.*" That's not how two's complement works. I mean yes, the first bit does tell you if it is negative, but it represents *more* than just that. – Nicol Bolas May 30 '22 at 15:38

1 Answers1

1

Normalized integer conversions are based on the bit-width of a number. An X-bit unsigned, normalized integer maps from the range [0, 2X-1] to the floating-point range [0.0, 1.0]. This is true for any X bit-width.

Signed, normalized conversion just uses two's complement signed integers and yields a [-1.0, 1.0] floating-point range. The only oddity is the input range. The two's complement encodes from [-2X-1, 2X-1-1]; this is an uneven range, with slightly more negative storage than positive. A direct conversion would make an integer value of 0 a slightly positive floating point value.

Therefore, the system converts from the range [-2X-1-1, 2X-1-1] to [-1.0, 1.0]. The lowest input value of -2X-1 is also given the output value -1.0.

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
  • If I read the components from the vertex input stream with VK_FORMAT_A2B10G10R10_SNORM_PACK32 the API does the conversion for me to float. However down the track if I want to read it from a uniform or storage buffer I'd have to do the conversion myself. It sounds a bit fiddly, does it actually involve a conditional branch to check if the value is negative? – Zebrafish Jun 01 '22 at 11:35
  • @Zebrafish: I would avoid trying to process such values at all if possible. There are GLSL functions for de-normalizing shorts or bytes (signed and unsigned), but not for 10-bit values. So just use shorts. Also, even if there is a conditional branch, [why do you think it would matter much?](https://stackoverflow.com/a/37837060/734069) – Nicol Bolas Jun 01 '22 at 13:21