half-precision 16-bit floating point
Most uses of 16-bit floating point are the ieee-754 binary16 aka half-precision floating point format, but other formats with different choices of exponent vs. significand bits are possible.
(However, related formats like Posit that have similar uses but a different binary format are not covered by this tag)
The floating-point tag wiki has links to more info, and lists other tags. (This tag was temporarily a synonym of precision, but should stay separate because half-precision is less widely implemented than float
/ binary32 and double
/ binary64.)
16-bit floating point has less precision (mantissa aka significand bits) and less range (exponent bits) than the widely used 32-bit single-precision IEEE754 binary32 float
or 64-bit binary64 double
. But it takes less space, reducing memory bandwidth requirements, and on some GPUs has better throughput.
It's fairly widely supported on GPUs, but on x86 CPUs at least, support is limited to conversion to/from float
. (And only on CPUs that support AVX and the F16C extension, e.g. Intel starting with IvyBridge.)
If any CPU SIMD extension supported math on half-precision directly, it would have twice the elements per SIMD vector and thus twice the throughput of float
for vectorizable tasks. But such support is not widespread in 2020 if it exists at all.