Max value here:
import numpy
numpy.finfo(numpy.float32).max
3.4028235e+38
Then I run code
a = np.array([0, 0, 0], dtype=np.float32)
for _ in range(100_000_000): a += 1.0
Why does it end up being
array([16777216., 16777216., 16777216.], dtype=float32)
?
I've asked GPT4 and this is the answer:
The reason your numpy array is maxing out at 16,777,216 is due to the precision limit of the float32 data type. This doesn't have to do with the maximum value a float32 can represent, but rather the minimum difference between two successive numbers a float32 can represent.
In your case, you're incrementing your array by 1 each time in a loop, but float32 can only represent integers accurately up to 16,777,216. Beyond this value, due to its precision limit, float32 cannot represent the next integer 16,777,217, so it rounds off to the closest number it can represent, which is still 16,777,216.
This property is inherent to how floating point numbers are represented in computer memory, it's a trade-off between range and precision. Although float32 can represent very large numbers, it cannot do so with very fine precision.
If you need to represent larger integers accurately, consider using int32 or int64 if you're only working with integers, or float64 if you need floating point numbers.