1

For a double data type it is stated in the docs, that it is consuming exactly 8 bytes, but for decimal, there is no such clear statement. In SQL Server - as comparison - it is stated, that decimal takes between 5 - 17 bytes depending on the chosen precision. see here.

How much space is decimal consuming in spark?

Dale K
  • 25,246
  • 15
  • 42
  • 71
Tarek Salha
  • 307
  • 3
  • 12
  • 1
    The docs clearly state that decimal is backed by Java's `BigDecimal`. It is an arbitrary-precision type that consists of a `BigInteger` scaled value and a 32-bit integer scale. `BigInteger` could be anything from a single byte to many bytes, so it all depends on the desired precision. – Hristo Iliev May 19 '22 at 07:10
  • Ok, I just read a bit through this [article](https://stackoverflow.com/questions/2501176/java-bigdecimal-memory-usage) and `decimal` appears to be huge compared to `double` or `float`. At least 10x the size for ordinary currency values. I really need to check, if I could switch to these ones instead. – Tarek Salha May 21 '22 at 08:01

0 Answers0