Why should you ever use a float instead of a double?

Question

I'm pretty new to coding, so this might be a stupid question, but why should you ever use a float when a double is more precise and has more bits?

Answer 1

Memory

The biggest reason is memory space. A float takes up 32 bits (4 bytes) while a double takes up 64 bits (8 bytes). This may not seem like a big deal, but some applications deal with a lot of numbers.

Let's say I'm simulating some system with a 3D grid that is a cube of values with a width of 1,000 points. That's 1,000,000,000 data points. If each point is a float value, I need 4 GB of memory. For doubles, I need 8 GB of memory. If I need to store more than one value at each point in the grid, the memory requirements just get bigger.

Grids similar to what I just described can be useful in physical simulations (climate or weather models, certain fluid dynamics simulations, etc). In those types of simulations, adding twice as many points can often be more useful than adding twice the precision at each point.

Speed

Depending on the hardware, the speed of calculation might also be a factor for time-sensitive calculations, but which is better depends on the underlying hardware. (See this answer for more details on that: https://stackoverflow.com/a/4584707/5941564)

If you are developing real-time software for a very specific piece of hardware that handles float faster than doubles, you should favor floats. But, some processors handle doubles faster, so you need to know your hardware before making that kind of performance decision.

Answer 2

I've got the opposite rule of thumb -- I have not needed many algorithms outside some heavy simulations which I compute with Octave that would require doubles over floats (save computing deep zoom Mandelbrot fractals and even those require arbitrary precision arithmetics after a certain point). Surely one needs to understand the constraints of the application and the constraints of the CPU architecture that one is working on.

I would often go even one step further, trying to avoid floating points, when it's possible to use 8-bit or 16-bit integers -- this apparently includes even Machine Learning applications, given that both Intel and Arm architectures have implemented 'neural network' extensions solely operating on 8-bit data and they have already implemented 16-bit float operations on hardware as well.

The smaller memory footprint of floats plays a crucial factor in four ways at least, three of which are related to speed. When the algorithm is memory bound (the number of arithmetic operations is small compared to the number of memory operation and the data does not fit in 1st level cache), float operations will theoretically be about twice as fast as doubles, as one needs to load and save half the memory.

When the amount of operations is small, but still larger than a few, many floating point algorithms will vectorise -- One can utilise SIMD architecture to put twice as many single precision floating point numbers to a register, executing 4 parallel floating point operations (or more per each starting CPU clock cycle) compared to 2 parallel double precision floating point operations.

Furthermore, the number of floating point numbers placed in registers may play an important role. Some algorithms may need to operate on 64, 128 or 256 different values at the same time -- spilling registers back and forth memory is costly. Some optimisation strategies may need that too (even though Out Of Order -execution could solve the problem automatically).

While modern computer architectures and even many embedded or low-power processors do contain floating point units, it's possible that they are still lacking double precision support completely, or its implementation is inherently much slower.

Finally the size it self plays a role in conserving memory on the 100 million customer devices, even if one as a developer does not care.